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I.   STATEMENT  OF  THE  PROBLEM 

During  the  past  fifty  years  there  have  been  dramatic 
advances  in  military  "hardware"  but  relatively  modest  changes 
in  personnel  management  techniques.   Although  the  need  to  keep 
pace  in  weapons  technology  is  not  disputed,  this  writer  believes 
that  the  potential  return  on  investment  available  from  improved 
personnel  management  has  escalated  sharply  in  the  past  few 
years--to  the  point  where  personnel  research  can  now  compete 
more  successfully  with  "hard"  research  for  scarce  defense 
resources.   This  belief  is  based  on  three  factors:   (1)  The 
contention  that  people  vice  hardware  are  usually  the  decisive 
factor  in  combat;  (2)  The  impact  of  the  All  Volunteer  Force 
concept  on  the  U.  S.  military;  and  (3)  The  implications  of 
the  Operational,  Technical,  Managerial  System  (OTMS)  for  the 
U.  S.  Navy. 

The  first  factor  is  mentioned  (in  a  business  vice  military 
context)  in  Thompson's  article  [62]  where  he  cites  the  view 
of  D.  C.  Burnham,  Chairman  of  Westinghouse  Electric  Corporation 
as  follows:   "Nearly  everyone  has  the  materials,  the  facilities, 
the  techniques  available  to  them,  but  how  they  use  these  things 
depends  on  their  people."   Admiral  Thomas  H.  Moorer  described 
the  issue  in  a  military  context  in  an  address  to  the  graduating 
class  at  the  Naval  Postgraduate  School  on  March  29,  1974.   He 
reported  that  analyses  of  major  naval  battles  of  World  War  II 
disclosed  that  a  majority  of  them  were  won  by  the  side  with 


technically,  and/or  numerically ,  inferior  forces.   His  con- 
clusion was  that  the  personnel  involved  had  made  the  crucial 
difference. 

The  second  factor  evolves  from  recent  changes  in  the  fabric 
of  U.  S.  society  which  have  brought  increasing  pressure  to 
bear  on  the  military  services  toward  increased  consideration 
of  individuals.   This  is  compounded  by  the  reality  of  the  All- 
Volunteer  concept  which  has  forced  the  military  services  into 
head- to-head  competition  with  industry  over  the  nation's  human 
resources.   The  "sanctuaries"  (draft,  etc.),  so  long  enjoyed 
by  the  military,  have  been  abolished,  and  this  has  thrust  the 
military  manpower  problem  onto  center  stage. 

The  third  factor  results,  in  part  at  least,  from  the  other 

two  which  have  begun  to  force  dianges  in  the  traditional  Naval 

Officer  career  patterns.   The  jack-of-all- trades ,  master-of- 

none  Naval  Officer  generalist  of  yesteryear  is  finding  it 

increasingly  difficult  to  cope  with  the  complex  and  varied 

problems  of  today's  Navy.   Bowling  [6]  expressed  it  thus: 

It  has  been  a  long-accepted  fact  that  the  Navy  needs 
both  good  operators  and  good  managers.   What  has  not  been 
accepted  or  fully  recognized  is  that,  in  the  current  tech- 
nological, strategic,  and  tactical  environment- -given  the 
complexities  of  nuclear  ships  and  weapons  systems,  coupled 
with  the  relatively  short  span  of  an  officer's  career- -it 
is  no  longer  possible  to  develop  both  within  the  same 
individual. 

He  further  recommended  that  the  CNO : 

.  .  .  establish  the  policy  that  all  unrestricted  line 
officers  specialize  in  either  Operations  or  Management, 
with  the  understanding  that  both  specialties  will  provide 
equal  but  parallel  paths  to  selection  for  flag  rank  in 
the  unrestricted  line. 
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Although  Bowling's  arguments   have  merit,  they  also  have 
drawbacks,  and  his  concept  of  sea-going  operators  and  shore- 
based  managers  did  not  come  to  pass.   Instead,  the  Operational, 
Technical,  Managerial  System  (OTMS)  was  established  in  the 
summer  of  1972.   The  concept,  and  the  rationale  behind  its 
development  were  described,  in  part,  as  follows  [107]: 

.  .  .  Although  there  are  several  approaches  which  could  be 
taken  to  provide  specialization  across  the  full  spectrum 
of  Navy  endeavor,  Admiral  Zumwalt  has  firmly  rejected  the 
idea  of  going  to  a  wet/dry  Navy.   (Writer's  note:   This 
was  Bowling's  proposal.)   Navies  of  other  nations  have 
tried  that  approach  and  found  it  unsatisfactory.   The 
Operational  Technical  Managerial  System  has  as  its  goal 
the  development  of  the  broad  sea  and  operationally-oriented 
unrestricted  line  officer  blended  with  the  technical  exper- 
tise normally  found  in  the  restricted  line  and  staff  corps 
.  .  .   Before  a  naval  officer  should  consider  becoming  a 
career  subspecialist  (concentration  within  a  subspecialty 
area) ,  the  officer  should  complete  at  least  two  operational 
tours  .  .  .   Through  a  system  of  evolution  we  are  imple- 
menting .  .  .  development  and  identification  systems  to 
meet  the  wide  spectrum  of  Navy  technical  and  managerial 
requirements . 

These  factors  illustrate  the  continuing  and  increasing 
need  for  selection  (hiring,  firing,  promoting),  development 
and  placement  within  the  U.  S.  Navy.   In  this  macro  context 
the  need  is  for  an  integrated  systems  approach  to  personnel 
management  involving  many  complementary  subsystems.   This 
study  focused  on  one  (proposed)  subsystem:   the  personnel 
assessment  center.   The  approach  used  was:   First,  to  research 
the  philosophies,  structures  and  applications  of  the  assess- 
ment center  concept,  and  evaluate  the  concept  in  the  are^s  of 
selection,  development  and  placement;  and,  next,  to  put  the 
assessment  center  in  perspective  within  an  "ideal"  personnel 
management  system,  explaining  the  interfaces  and  relationships 
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between  the  assessment  center  and  other  subsystems;  and, 
finally,  to  develop  a  research  proposal  for  the  design  of  a 
prototype  personnel  assessment  center  for  the  management  cur- 
riculum of  the  Naval  Postgraduate  School. 

Hardesty  and  Jones  [31]  offer  their  opinion  as  to  the 
potential  value  of  the  assessment  center: 

Because  the  concern  of  the  assessment  center  is  with 
the  organization's  leaders  of  the  future,  th'e  ultimate 
validity  of  this  program  .  .  .  may  well  be  measurable  in 
the  future  success  of  the  companies  as  contrasted  to  sim- 
ilar companies  who  do  not  make  such  an  effort.   The  stakes 
are  high,  but  on  the  other  hand  they  always  have  been  .  .  . 

If  the  stakes  are  considered  to  be  high  in  the  business 

world,  can  they  be  otherwise  in  the  military?   The  professional 

military  man  is  acutely  aware  of  the  consequences  of  finishing 

second  in  combat.   There  will  always  be  a  need  for  good  men 

and  women--the  problems  are  finding,  developing  and  retaining 

them.   This  paper  is  devoted  to  the  examination  of  one  proposed 

solution  to  those  problems:   the  personnel  assessment  center. 
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II.   BACKGROUND 

A.   DEFINITIONS 

A  typical  industrial  assessment  center  is  described  by 

Byham  [11]  as  follows: 

In  these  centers,  specially  trained  managers  (and  occasion- 
ally psychologists)  act  as  "assessors"  who  evaluate  candi- 
dates for  promotion- -either  into  management  or  within 
management- -on  their  potential  and  their  areas  of  weakness. 
Groups  of  men  pass  through  series  of  standardized  exercises 
such  as  management  games,  in-basket  tests,  and  leaderless 
discussion  sessions,  while  assessors  observe  their  behavior 
closely. 

The  assessors  discuss  each  candidate's  performance 
separately  and  then  generate  a  comprehensive  report  on  each 
candidate  which  management  can  combine  with  current  per- 
formance information  as  it  sees  fit.   As  well  as  identifying 
the  men  most  likely  to  succeed,  the  assessment  reports  spell 
out  the  individual  deficiencies  of  each  candidate  and  suggest 
guidelines  for  management  to  use  in  developing  him. 

The  U.  S.  Army  Infantry  School  (USAIS)  Assessment  Center 

[127]  uses  this  definition: 

The  term  assessment  center  is  used  to  describe  a  process  by 
which  candidates  for  a  position  are  put  through  a  series  of 
standardized  exercises  designed  to  simulate  the  conditions 
of  that  position  and  to  show  if  they  have  the  skills  and 
abilities  necessary  to  perform  it. 

The  assessment  process  is  conducted  using  a  series  of 
situational  exercises  designed  to  measure  leadership  behav- 
iors and  skills  one  level  higher  than  the  individual  has 
performed.   Officer  candidates  are  cast  in  the  role  of 
2nd  lieutenants,  lieutenants  as  company  commanders,  and 
senior  NCOs  as  platoon  leaders.   In  addition,  a  battery  of 
psychological  tests  are  used  to  measure  attitudes,  mental 
ability,  reading  skills  and  interests.   Peer  ratings  and 
self  ratings  combined  with  those  of  the  trained  assessment 
staff  provide  a  trifocal  view  of  each  individual  tested. 

As  indicated  by  the  USAIS  definition,  the  term  assessment 

center  more  properly  refers  to  a  process  or  program  rather 
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than  to  a  physical  location.   In  fact,  some  "assessment  centers" 
are  taken  to  various  corporation  field  sites  for  administration 
vice  transporting  candidates  to  an  "assessment  center"  facility. 

B.   HISTORY 

Various  articles  have  traced  the  genesis  of  the  assessment 

center  to  the  Bible.   Wilson  and  Tatge  [70]  reference  the 

advice  which  Jethro  gave  to  Moses  on  selecting  his  rulers: 

Moreover  thou   shalt  provide  out  of  all  the  people,  able 
men,  such  as  fear  God,  men  of  truth,  hating  covetousness ; 
and  place  such  over  them  to  be  ruler  of  thousands,  and 
rulers  of  hundreds,  rulers  of  fifties,  and  rulers  of  tens. 

Jaffee  [35]  cites  Judges,  Chapter  7  as  the  first  use  of 

situational  techniques: 

.  .  .  the  Lord  provided  Gideon  with  a  means  of  choosing  the 
best  among  his  men.   "Bring  them  down  unto  the  water,  and 
I  will  try  them  for  thee  there  ....   Separate  every  one 
that  lappeth  of  the  water  with  his  tongue,  as  a  dog  lappeth, 
him  shalt  thou  set  by  himself;  likewise  every  one  that 
boweth  down  upon  his  knees  to  drink  .  .  .  ." 

From  an  initial  force  of  32,000  men,  Gideon  was  able  to 
select,  by  such  techniques, an  elite  group  of  300  and  routed  a 
numerically  superior  army. 

Prior  to,  and  during,  World  War  II  the  Germans  used  multiple 

assessment  techniques  in  officer  selection.   The  German  efforts 

inspired  the  British  War  Officer  Selection  Boards  (WOSB) ,  which 

in  turn  influenced  the  program  of  the  Office  of  Strategic 

Services  (OSS)  in  this  country.   The  other  primary  influence 

in  the  OSS  program  was  the   presence  of  Dr.  Henry  Murray  of 

Harvard  who  brought  with  him  concepts  derived  from  his  multiple 

assessments  in  personality  research  [97] .   Assessment  of  Men  [99] 

by  the  OSS  Assessment  Staff  is  a  detailed  and  very  readable 

account  of  this  effort. 
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All  of  the  programs  mentioned  thus  far  were  designed  for 
the  purpose  of  selection,  however  the  wartime  environment 
afforded  neither  time  nor  opportunity  for  proper  validation. 

Following  the  war,  only  the  British  retained  operational 
assessment  programs.   Their  Civil  Service  Selection  Boards 
(CISSB)  began  validation  efforts,  while  the  WOSB  evolved  into 
the  Royal  Commissions  Board  (RCB)  of  today. 

As  West  Germany  began  rebuilding  her  armed  forces,  the 
Federal  German  Navy  established  an  extremely  sophisticated  six- 
week  assessment  program  for  its  officers  [57] .   This  program 
has  recently  been  merged  into  a  tri-service  staff  officer 
course. 

Since  the  close  of  the  OSS  centers,  the  U.  S.  military 
has   only  dabbled  sporadically  in  assessment  center  research. 
The  latest  effort  was  launched  in  1973  when  the  U.  S.  Army 
chartered  a  one-year  evaluation  of  a  pilot  personnel  assessment 
program.   This  pilot  program  was  established  at  the  U.  S.  Army 
Infantry  School  (USAIS) ,  Fort  Benning,  Georgia. 

In  addition  to  the  British  and  Germans,  the  Australian 
Army  and  Israeli  armed  forces  are  known  to  have  assessment 
programs . 

Although  the  U.  S.  military  has  lagged  other  nations  in 

the  use  of  assessment  centers  since  WW  II,  U.  S.  industry  has 

pioneered  in  the  field.   Dr.  Douglas  Bray  of  AT§T  set  up  an 

assessment  program  within  the  Management  Progress  Study  in 

1956.   Dr.  Bray  [7]  describes  this  study  as  follows: 

Although  the  Management  Progress  Study  was  instituted  as  a 
long-term  study  without  any  expectation  of  immediate 
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practical  results  .  .  .  .  ,  the  mere  conduct  of  the  study 
and  the  reporting  of  gross  observations  on  the  total  group 
of  subjects  have  led  to  significant  changes  in  certain 
personnel  practices  .  .  .   Among  them  are  ....  the 
application  of  the  assessment  center  method  to  the  selection 
and  development  of  managers. 

The  Management  Progress  Study  was  designed  as  a  ten-year 
longitudinal  study- -and  indeed  it  was  continued  to  fruition- - 
but  the  merits  of  the  assessment  center  were  quickly  recognized 
and  the  first  operational  assessment  center  in  the  AT§T  system 
opened  its  doors  in  1958.   Following  AT&T's  lead,  other  U.  S. 
firms,  such  as  :   IBM,  General  Electric,  J.  C.  Penney,  Standard 
Oil  (Ohio)  and  Sears,  established  assessment  centers,  and  the 
program  spread  to  other  countries. 

The  influence  of  the  AT§T  program  has  been  considerable 
and  most  of  the  subsequent  assessment  programs  bear  at  least 
partial  resemblance  to  it. 

C.   OVERVIEW 

Although  all  assessment  programs  emanate  from  the  same 
basic  concepts,  they  differ  considerably  along  certain  dimen- 
sions, notably: 

1.  Objectives  (Selection,  development,  placement  or  research) 
Most  of  the  applications  are  for  selection,  however  a 

growing  number  have  development  as  a  secondary  objective. 

2 .  Program  Philosophies 

Assessment  philosophies  range  along  a  continuum  from 
psychometric  to  behavioral- -or  in  another  sense,  from  signs  to 
samples.   There  are  a  few  programs  at  either  extreme,  however 
most  fall  nearer  the  center  and  incorporate  both  signs  and 
samples  . 
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3.  Program  Lengths 

These  vary  from  one  day  to  six  weeks.   The  majority 
are  five  days;  with  two  and  one-half  to  three  days  of  assess- 
ment followed  by  two  to  two  and  one-half  days  of  evaluation 
and  report  writing  by  the  assessors. 

4.  Assessment  Variables 

Since  most  programs  are  selection-oriented,  the  big 
emphasis  has  been  on  stable  variables.   If  development  were 
a  program  objective,  unstable  variables  would  have  to  be  included 
to  identify  those  areas  offering  a  viable  training  potential. 
Numbers  of  variables  assessed  vary  from  ten  to  fifty-two. 
Howard,  in  her  survey  of  current  assessment  programs,  [34] 
found  the  following  dimensions  to  be  the  most  popular: 
"(a)  leadership,  (b)  organizing  and  planning,  (c)  decision 
making,  (d)  oral  and  written  communications  skills, (e)  ini- 
tiative, (f)  energy,  (g)  analytical  ability,  (h)  resistance 
to  stress,  (i)  use  of  delegation,  ( j )  behavior  flexibility, 
(k)  human  relations  competence,  (1)  originality,  (m)  controlling, 
(n)  self-direction,  and  (o)  overall  potential." 

5.  Program  Content  (Instruments  and  techniques). 

The  content  of  the  program  should  be  an  extension  of 
the  program  philosophy.   The  listing  which  follows  will  proceed 
from  signs  to  samples  and  will  be  as  inclusive  as  possible, 
vice  representative  of  any  single  program: 

a.  Paper  and  pencil  tests  of  attitudes,  interests, 
mental  ability,  and  achievement  (math,  reading  grammar,  etc.). 

b.  Projective  personality  tests  such  as  sentence 
completion,  adjective  check-lists,  etc.. 
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c.  Interviews  by  assessors.   Some  programs  also  have 
assessees  interview  other  role-playing  assessees  to  evaluate 
their  interviewing  ability. 

d.  Written  assignments  such  as  essays.   These  may  be 
solicited  prior  to  arrival  at  the  center.   The  topic  may  be 
of  the  candidate's  choice,  or  assigned  to  him. 

e.  Simulations  may  be  either  individual  (the  in-basket 
for  instance)  or  group  (such  as  leader  less  -group  discussions, 
management  games,  war  games,  etc.). 

The  in-basket  is  a  widely-used  individual  simulation 
with  high  face  validity  and  a  concomitant  training  effect.   It 
consists  of  a  series  of  notes,  memos,  and  letters  typical  of 
the  in-basket  of  the  position  for  which  the  candidate  is  being 
evaluated.   The  scenario  imposes  a  time  constraint  and  forces 
the  assessee  to  work  alone  without  recourse  to  other  individuals. 
He  is  provided  ample  information  on  the  role  he  will  play,  (such 
as  an  organizational  chart,  pertinent  personal  data  on  key 
associates,  etc.)  and  is  asked  to  put  all  of  his  responses  in 
writing.   He  may  delegate  items  to  subordinates,  take  action 
himself,  defer  action  until  further  information  is  obtained, 
schedule  meetings,  or  write  letters,  as  he  sees  fit.   Most 
programs  schedule  an  interview  session  following  the  in-basket 
where  an  assessor  can  question  the  candidate  on  the  motives 
behind  his  actions.   Evaluations  of  the  in-basket  range  from 
informal  subjective  impressions  to  highly-structured  objective 
analyses . 

Leaderless  group  discussions  typically  assign  the 

candidates  differing  points  of  view  which  they  are  to  defend 
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individually.   Following  this  discussion,  they  are  asked  to 
rank-order  the  positions.   Since  the  points  of  view  are  designed 
to  be  equal  in  desirability,  the  candidate  who  is  most  per- 
suasive, influential  and  accepted  by  the  group  should  emerge 
with  the  "winning"  position. 

The  various  "games"  used  in  assessment  programs 
can  provide  evaluations  of  leadership  ability,  interpersonal 
skills,  decision  making  ability,  organizational  ability  and 
resistance  to  stress- -depending  on  the  particular  objectives 
of  the  program. 

6.   The  Assessors 

Several  sources  state  that  familiarity  with  the  position 
sought  by  the  candidates  is  the  single  most  important  attribute 
of  an  assessor.   This  probably  explains  why  most  programs  uti- 
lize line  managers  two  to  three  levels  above  the  "assessment 
position"  as  assessors.   A  few  programs  employ  clinical  psy- 
chologists to  interpret  the  projective  tests,  but  very  few  have 
psychologists  actively  involved  in  the  assessment  and  evaluation 
processes.   Assessor/assessee  ratios  range  from  1:1  to  1:4  and 
length  of  assignment  as  an  assessor  varies  from  one  program 
(one  to  two  weeks)  to  six  to  eight  months  or  longer.   Assessor 
training  periods  may  be  as  short  as  a  few  days  or  as  long  as 
two  to  three  weeks.   The  training  may  include  lectures,  films, 
etc. ,  but  the  most  effective  and  widely-used  training  technique 
is  to  involve  the  trainee  in  actual  assessments  on  a  "norm 
group."   AT£T  uses  successful  incumbents  of  the  position  being 
sought  by  candidates  as  the  "norm  group."   This  provides  assessor 
trainees  with  two  benefits:   (1)  a  tangible  example  of  performance 
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by  organizationally  defined  "successes,"  and  (2)  an  opportunity 
to  err  while  learning  without  jeopardizing  a  candidate's  career. 
A  point  upon  which  there  is  unanimous  agreement  is  that  well- 
trained  assessors  are  an  absolute  prerequisite  of  an  effective 
assessment  program. 

7.  The  Evaluations 

After  the  assessments  are  completed,  the  assessors 
document  their  observations.   A  meeting  is  then  held  where  can- 
didates' records  are  reviewed  individually  and  ratings  made 
by  the  assessors  on  the  basis  of  all  available  information. 
Any  disparities  in  the  ratings  are  discussed  further,  however 
few  organizations  require  consensus  (e.g.,  that  disparate 
ratings  be  changed).   The  final  result  may  be  "pass-fail"  recom- 
mendations to  line  management,  or  merely  reports  of  the  assessors' 
evaluations.   These  reports  are  usually  in  narrative  form  (con- 
sidered best  for  use  by  laymen) ,  however  some  firms  prefer 
profiles  consisting  of  graphs  or  scales. 

8.  Uses  of  the  Data 

Most  programs  are  selection  oriented,  however  final 
selection  decisions  are  not  made  by  the  assessors  (with  the 
possible  exception  of  initial  hiring  situations  where  job 
performance  data  is  not  available,  and  the  Federal  German  Navy 
program).   In  most  programs,  assessment  evaluations  are  furn- 
ished to  line  management  where  it  can  be  integrated  with  other 
information,  such  as  performance  appraisal  data,  as  the  managers 
see  fit.   In  such  cases  (the  majority)  selection  decisions  lie 
solely  with  line  management. 
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Development  applications  are  few,  but  increasing  in 
numbers.   They  generally  provide  the  candidate  with  knowledge 
of  his  strengths  and  weaknesses.   The  "ultimate"  outcome  would 
be  a  comprehensive  development  plan  specifically  tailored  to 
the  individual. 

Programs  with  placement  as  an  objective  are  rare.   The 
Sohio  program  includes  placement  as  an  objective  and  is  described 
by  FJiil:]c  and  Jones  [85]  s  .. 

9.   Feedback  to  Candidates 

Feedback  is  a  part  of  most  programs  - -including  selection 
oriented  ones- -even  though  it  represents  a  developmental  function. 
It  may  be  provided  to  all  candidates,  or  only  to  those  who 
request  it.   Feedback  may  be  given  by  lay  assessors,  psychologists 
or  line  managers- -as  the  organization  prefers.   It  is  usually 
oral,  confined  to  observed  behaviors  and  test  performance,  and 
provides  the  candidate  with  recommendations  for  a  personal 
development  plan. 
10.   Summary 

Bender  [3]  provides  an  excellent  overview  summary.   He 
conducted  a  survey  of  organizations  operating  assessment  centers 
and  obtained  data  from  thirty-four  operational  programs.   The 
results  of  his  survey  are  presented  in  the  following  figures 
and  provide  a  good  picture  of  the  current  character  of  assess- 
ment in  the  United  States  and  Canada. 
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Figure  1. 

Primary  Evaluation  Devices  Used  by  Companies 
Operating  Assessment  Centers 


In-basket  exercises  used. 
Business. games  exercises  used. 
Assigned  roles 
Nonassigned  roles 
Leaderless  group  discussion  exercises 
used. 

Assigned  roles 
Nonassigned  roles 
Films  used. 
Video  tapes  used. 
Psychological  tests  used. 
For  the  most  part,  assessment  devices 

are  locally  produced. 
For  the  most  part,  assessment  devices 

are  purchased  externally. 
An  in-depth  background  interview  is 
given  to  assessees. 

Source:   Bender  [3] 
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Figure  2. 

Operating  Procedures  Followed  in  Administration 
of  Assessment  Centers 

Not 
Yes    No     Checked 

Assessees  are  nominated  by  supervision.  30     4 
Assessees  nominate  themselves  for 

assessment.*  9     23       2 
A  full-time  director  administers  the 

assessment  center.  11    23 
A  part-time  director  administers  the 

assessment  center.  19    15 
The  assessment  center  director  is  an 

industrial  psychologist.  11    23 

Females  are  assessed.**  25     9 

Minorities  are  assessed.**  29     4       1 

Homework  is  assigned.  13    21 
Assessee's  performance  is  considered 

as  pass  or  fail.  3    31 

Female  assessors  are  used.  13    20       1 
Assessment  is  conducted  off-site  and 

away  from  the  work  environment.  2  5     9 
Assessors  rotate  in  viewing  assessees 

during  evaluation.  29     5 
Assessee  performance  is  video-taped  for 

use  in  evaluation.  11     23 

Re-assessment  of  assessees  is  permitted.  19    15 
A  checklist  for  evaluation  is  used  by 

assessors  during  evaluation.  24    10 

Peer  evaluations  are  made  by  assessees.  24    10 
Assessment  center  has  been  validated 

against  performance  criteria.  16    18 

*Two  companies  considering  for  future  evaluation. 
**0nly  one  organization  indicated  that  it  had  not  yet  assessed 
either  females  or  minorities,  but  it  added  that  no  restrictions 
existed  for  not  doing  so. 

Source:   Bender  [3] 
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Figure  3. 
Operating  Characteristics  of  Assessment  Centers 

Number  of  assessees  evaluated  when  your  assessment  center  operates: 

.A    £  10  11  li  M 

Response  frequency    13   2    2   11    2    3 
Levels  assessed  by  your  assessment  (enter:* 

First  Line   Middle  management   Upper  management   Newly  hired 
24  17  3  7 

Number  of  days  your  assessment  center  operation  takes: 

1   1.5    2   ^5   3   3^5   4   _5   6 
Response  frequency    T     2  TO"  I    71    Till 

Total  number  of  evaluation  exercises  used  during  the  operation 
of  your  assessment  center: 

i:567_891_0ni5±0 
Response   frequency  763411        2        2        1        1 

Number  of  years  your  assessment  center  has  been  in  operation: 

o-i  ^isisesio^rs 

Response  frequency:      1   12   731231    1    1 
Number  of  employees  you  have  assessed  to  date:+ 

0   0-500   500-1000   1000-3000   75000 
Response  frequency:    1     25        5  2      1 

*Several  companies  indicated  that  their  assessment  center  was 
used  for  evaluation  of  more  than  one  level  of  supervision. 
+One  center  was  not  yet  operational. 

Source:   Bender  [3] 
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Figure  4. 

Evaluation  Parameters  Most  Frequently  Observed  in 

Assessment  Centers 


Parameter 

Impact 

Energy 

Forcefulness 

Perception 

Creativity 

Range  of  interests 

Salesmanship 

Listening  skill 

Risk-taking 

Independence 

Management  control 

Use  of  delegation 

Problem  analysis 

Leadership 

Oral  communication  skills 

Oral  presentation  skills 

Written  communication  skills 

Organizing  and  planning 

Decision-making  skills 

Inner  work  standards 

Resistance  to  stress 

Attitude  toward  peers 

Attitude  toward  subordinates 

Attitude  toward  superiors 

Self -evaluation 

Behavioral  flexibility 

Source:   Bender  [3] 
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Figure  5. 

Uses  Made  of  Performance  Data 
Generated  by  Assessment  Centers 


Assessees  are  provided  immediate 

feedback  on  their  performance. 
Feedback  is  given  orally  to  the 

assessee. 
Feedback  is  given  in  writing  to 

the  assessee. 
Feedback  is  given  through  the  line 

organization. 
Feedback  is  given  assessee  only  by 

assessment  center  personnel. 
Assessee's  evaluation  is  made 

available  to  top  management. 
Evaluation  results  are  used  to 

prepare  a  formal  plan  of  develop- 
ment for  the  assessee.  23     9 
A  follow-up  of  an  assesse's  continued 

development  after  assessment  results 

are  reported  is  a  routine  function  of 

assessment  center  activities.  14    17 
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Figure  6. 
A  Summary  of  Key  Characteristics  of  Assessment  Centers 

In-basket  exercises  are  used. 

Games  and  leaderless  group  discussions  (with  both  assigned  and 

nonassigned  roles)  are  used.  ' 

Psychological  tests  are  used  in  conjunction  with  simulated 

exercises . 
Assessment  instruments  are  locally  produced. 
In-depth  interviews  are  conducted. 
Assessees  are  nominated  by  supervision. 
Four  simulation  exercises  are  used  during  evaluation. 
Assessment  centers  are  operated  by  part-time  directors. 
Females  are  assessed. 
Minorities  are  assessed. 
Homework  is  assigned. 

Assessment  is  conducted  away  from  the  work  environment. 
Assessors  rotate  in  viewing  assessees  during  evaluation. 
Re-assessment  is  permitted. 
Peer  evaluations  are  made  by  assessees. 
One  assessor  for  one  assessee  is  used. 
A  maximum  of  six  people  are  assessed  at  one  time. 
Assessors  are  two  organizational  levels  above  assessees. 
Assessment  centers  operate  for  three  days. 
Assessee' s  performance  is  rated,  not  ranked. 
Rating  systems  are  based  on  research. 
Parameters  assessed  depend  on  the  purpose  of  the  assessment 

center. 
Assessees  receive  immediate  oral  feedback  on  their  performance 
Assessees  receive  written  feedback  on  their  performance. 
Feedback  is  disseminated  by  assessment  center  personnel. 
Assessment  center  results  are  used  for  formal  development 

plans . 

Source:   Bender  [3] 
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III.   APPROACH 

A.   INTERVIEWS 

Working  within  time  and  fund  constraints,  four  sites  were 
visited.   These  included  two  military  (U.  S.  Army)  applications, 
an  AT§T  (Pacific  Bell  Telephone)  assessment  center  and  an 
assessor  training  session  for  a  one-day  behaviorally-oriented 
program.   A  brief  summary  of  the  visits  with  lists  of  signifi- 
cant aspects  of  each  site  and  application  follows. 

1.   United  States  Army  War  College  (USAWC) ,  Carlisle  Barracks, 
Pennsylvania 

This  visit  afforded  the  opportunity  to  interview  indi- 
viduals who  were  involved  in  the  USAWC  "Study  of  Leadership  for 
the  Professional  Soldier"  [128],  [129],  [130],  and  [131];  namely, 
Dr.  Donald  D.  Penner  and  LTCOL  Dandridge  M.  Malone.   In  addition 
to  providing  copies  of  the  references  cited  above,  and  amplifying 
information  on  the  study,  they  outlined  some  of  the  Army's  other 
activities  which  preceded  the  USAIS  assessment  program. 

The  influence  of  the  Center  for  Creative  Leadership  in 
Greensboro,  North  Carolina  (Olmstead,  et. al. [122]  refers)  was 
explained.   When  the  Army  became  seriously  interested  in  assess- 
ment centers,  a  few  years  ago,  twelve  Brigadier  Generals  were 
sent  through  the  Center  for  Creative  Leadership  to  determine 
their  reaction  to  an  assessment  program.   This  program  was 
purported  to  contain  several  paper  and  pencil  tests  and  one 
complex  simulation.   The  simulation  is  of  particular  interest. 
It  featured  a  scenario  set  in  the  31st  century,  and  required 
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candidates  to  cycle  through  six  defined  leadership  roles.   The 
futuristic  setting  was  chosen  to  eliminate  any  possibility  of 
bias  from  previous  candidate  experience.   The  entire  simulation 
was  observed  by  assessors  through  "one-way"  mirrors',  and  also 
video-taped.   At  intervals,  action  was  halted  and  candidates 
were  asked  to  record  their  perceptions  of  the  exercise  at  that 
point.   This  simulation  lasted  an  entire  day,  and  was  followed 
by  a  detailed  feedback  which  stresscr1  the  comparison  between 
the  individual's  self -perception  and  the  perception  of  him  by 
others. 

Because  this  program  is  oriented  toward  research  and 
self -development ,  the  Center  does  not  provide  any  assessment 
information  to  a  candidate's  parent  organization.   It  does, 
however,  provide  transcripts,  video  tapes  and  other  data 
obtained,  directly  to  the  candidate.   The  twelve  Brigadiers 
who  underwent  assessment  were  reported  as  enthusiastic  in  their 
endorsement  of  the  program  as  a  vehicle  of  personal  inventory 
and  development. 

LTCOL  Quay  Snyder  [124]  offered  much  detail  on  the 
AWC  Inventory  and  Assessment.   His  review  of  relevant  instru- 
ments and  information  on  source  material  was  especially  valuable. 

Probably  the  greatest  benefit  from  the  AWC  visit  derived 
from  the  constructive  criticisms  of  this  writer's  concepts  of 
an  assessment  program,  and  the  alternative  suggestions  offered. 
Though  too  numerous  to  acknowledge  individually,  there  were 
many  seeds  which  took  root  and  flourished  during  the  ensuing 
research  period.   The  results  provide  much  of  the  substance 
of  this  thesis  . 
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Some  of  the  more  significant  factors  were: 

a.  The  concept  of  more  than  one  assessment  during  the 
course  of  a  career. 

b.  The  "systems  approach"  to  personnel  management 
which  views  assessment  centers  as  a  subsystem  of  the  larger 
system. 

c.  The  USAWC  Leadership  Study. 

d.  An  as&.es sment  program  , ap.pl ied  in  a  military  environ- 


ment. 


ment 


e.   An  assessment  program  devoted  exclusively  to  develop- 


2 .   United  States  Army  Infantry  School  (USAIS)  Assessment 
Center  Pilot  Project 

This  operation  represents  a  current  military  application 
of  the  "ATqT  model,"  e.g.,  a  program  which  utilizes  a  mix  of 
paper  and  pencil  tests  and  experiential  instruments.   The  visit 
afforded  interviews  with  the  director,  COL  Wallace  Veaudry; 
assistant  director  LTCOL  Kenneth  Smith;  staff  members,  MAJ  John 
Campbell  and  MAJ  Richard  Davis;  and  Dr.  Kay  Smith  of  the  Army 
Research  Institute  (ARI)  Field  Unit  which  is  co-located  with 
the  assessment  center. 

The  USAIS  assessment  program  entails  forty  hours  of 
assessment  in  a  three-day  period.   This  heavy  schedule  is  a 
result  of  the  inclusion  of  "competing"  instruments  for  evalu- 
ation.  At  the  completion  of  the  evaluation,  the  program  will 
be  factor  analyzed  and  those  instruments  with  little  or  no 
unique  contribution  will  be  eliminated.   This  should  stream- 
line the  program  to  a  more  realistic  schedule. 
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A  unique  feature  of  this  visit  was  the  fact  that 
current  staff  members  had  been  involved  in  the  design  and 
implementation  as  well  as  the  operation  of  the  assessment 
program.   Because  of  this,  staff  members  were  exceptionally 
understanding  and  knowledgeable  concerning  the  focus  and 
requirements  of  this  study.   Additional  background  on  U.  S. 
Army  assessment  center  research  was  provided,  and  an  assess- 
ment center  research  program  conducted  at  Fort  McClellan, 
Alabama  circa  1965  was  reported.   It  was  described  as  a  very 
expensive  operation  where  candidates  were  assessed  one  at  a 
time,  thus  the  assessor/candidate  ratio  was  something  like 
12:1.   In  contrast,  the  present  USAIS  ratio  is  about  11:18, 
which  is  more  representative  of  ratios  in  current  industrial 
assessment  programs.   The  use  of  video  tape,  and  the  structure 
and  content  of  the  counseling  feedback  were  other  facets  of 
the  program  worthy  of  mention. 

In  summary,  the  following  items  were  of  particular 
interest: 

a.  Information  concerning  assessment  center  design, 
and  implementation- -as  well  as  operation.   (Olmstead  et. al. 
[122]  pertains  to  design  of  this  program) . 

b.  Simulations  tailored  to  military  situations. 

c.  Use  of  video-tape  in  analysis  and  counseling. 

d.  Feedback  techniques  and  counseling  training. 

e.  Differences  between  programs,  reflecting  different 
levels  of  candidates  and  positions  being  assessed  for,  e.g., 
differences  in  course  content  between  the  following: 
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Course  Candidate  Level 

Officer  candidate  course  E5 

Advanced  NCO  course  E6-E7 

Basic  Officer  course  LT 

Advanced  officer  course  CPT 

f.   Assessor  training  and  employment.   (Assessors  are 

assigned  for  a  normal  tour  of  duty- -about  two  years- -therefore 

are  long-term  in  contrast  with  short-term  assessors  prevalent 

in  industry  programs) . 

3.   Dr.  Cabot  L.  Jaffee,  Florida  Technological  University, 
Orlando,  Florida 

The  visit  coincided  \\rith  an  assessor  training  session 
which  Dr.  Jaffee  was  conducting  for  selected  civil  service 
personnel  of  the  Naval  Training  Center,  Orlando.   The  training 
was  in  preparation  for  a  one-day  assessment  program  composed 
of  behavioral  instruments  (simulations,  etc.).   This  provided 
an  opportunity  to  observe  assessor  training  which  included  the 
administration  and  scoring  of  "the  in-basket  exercise  included 
in  the  assessment  program. 

Dr.  Jaffee  has  extensive  experience  in  the  field  of 
assessment  centers  and  has  contributed  much  to  the  literature 
[16],  [35],  [36],  [89]  and  [90].   Because  of  this,  his  comments, 
criticisms  and  suggestions  were  particularly  valued. 

Highlights  of  the  visit  were: 

a.  Observation  of  assessor  training. 

b.  Views  of  a  prominent  consultant  in  the  field  of 
assessment  centers. 
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4 .   Personnel  Assessment  Review  (PAR)  Program,  Pacific 
Telephone  Company,  Oakland,  California 

Because  of  the  great  influence  of  the  "AT§T  model"  on 
subsequent  assessment  programs,  a  visit  to  a  Bell  system  program 
was  given  high  priority.   During  the  interview  attention  was 
focused  on  operational  aspects  rather  than  design  or  theory. 

Salient  points  of  the  program  included: 

a.  Clinical  combination  of  variables  in  the  evaluation 
process . 

b.  Practice  of  having  candidates'  supervisors  observe 
the  evaluation  process. 

c.  Pre-assessment  interviews  done  by  assessors  at 
candidate's  organization. 

d.  Feedback  to  candidates  done  by  assessors  at  candi- 
date's organization. 

e.  Practice  of  mixing  men  and  women  in  groups  vice 
assessing  them  separately. 

B.   LITERATURE  REVIEW 
1.   Overviews 

This  section  is  provided  as  a  guide  for  the  reader  who 
seeks  additional  overview  information.   It  does  not  contain 
descriptive  data,  per  se,  but  rather  directs  the  reader  to 
selected  articles  in  the  literature.   Those  readers  not 
desiring  such  referral  advice  may  proceed  directly  to  section 
III-B-2,  Current  Applications. 

Several  articles  in  the  literature  provide  excellent 
overviews.  These  will  be  cited  in  the  following  order:  (1) 
General  articles  covering  a  wide  range  of  programs,  (2)  Articles 
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dealing  with  a  particular  type  of  program  (e.g.,  the  five-day 
program  patterned  after  the  ATfjT  model,  etc.)>  (3)  Articles 
dealing  with  specific  organizational  programs. 

An  article  in  Business  Week  [68]  provides  a  broad 
survey  of  the  field  from  one-day  assessment  programs,  to  the 
"five-day"  programs  employed  by  larger  firms  such  as  AT§T, 
J.  C.  Penney,  Standard  Oil  (Ohio),  etc.  and  gives  some  rough 
cost  figures.   Howard  [34]  presents  an  excellent  aggregation 
of  information  from  the  entire  field.   Cohen  and  Jaffee  [16] , 
although  primarily  concerned  with  government  applications, 
cover  most  of  the  significant  developments  in  the  field.   They 
also  provide  some  predictions  on  future  growth  and  changes  in 
government  assessment  center  applications. 

Slevin  [59]  describes  a  program  typical  of  those  found 
in  the  major  firms  employing  assessment  centers  (e.g.,  AT§T, 
Sears,  IBM,  General  Electric,  J.  C.  Penney,  Standard  Oil  of 
Ohio,  etc.).   This  article  has  a  particularly  comprehensive 
coverage  of  the  subject,  touching  briefly  on  most  of  the 
major  topical  areas.   This  writer  considers  Slevin's  predic- 
tions for  the  future  of  assessment  centers  to  be  particularly 
perceptive  and  well-founded.   Other  authors  writing  on  assess- 
ment programs  of  this  type  include:   Albrook  [1],  Byham  [11], 
and  Byham  and  Pentecost  [12] . 

McConnell  [43]  describes  a  one  day  assessment  program 
in  a  small  company.   McConnell  [44],  [45]  and  McConnell  and 
Parker  [46]  discuss  the  one  day  "package"  assessment  program 
available  from  the  American  Management  Association  (AMA) . 
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Information  on  the  AT§T  (Bell  System)  programs  is 
abundant  in  the  literature.   Articles  providing  general  data 
include  Jaffee  [35],  Jelks  [37],  and  Wikstrom  [69]. 

The  Standard  Oil  of  Ohio  (Sohio)  program  is  treated 
in  detail  in  the  book  by  Finkle  and  Jones  [85] .   Chapter  four 
of  the  book  contains  the  overview.   This  program  is  also 
described  by  Hardesty  and  Jones  [31] . 

Veaudry  and  Campbell  [65]  describe  the  USAIS  assess- 
ment center  pilot  project,  while  Snyder  [124]  covers  the  USAWC 
developmental  assessment  program.   Information  on  foreign 
military  programs  are  found  in  the  USAIS  unpublished  report 
of  a  trip  to  the  RCB  [126]  and  Schmahling's  [57]  article  on 
the  Federal  German  Navy's  program. 
2 .   Current  Applications 

a.   Industry 

It  is  difficult  to  obtain  an  accurate  estimate  of 

the  number  of  assessment  centers  in  existence,  however  the 

literature  provides  a  picture  of  increasing  growth  in  this 

area.   Pomerleau  [53]  notes: 

.  .  .  AT§T  affiliates  .  .  .  currently  operate  over  50 
centers  throughout  the  country  processing  upwards  of 
10,000  candidates  a  year  at  an  annual  cost  of  $5  million 
.  .  .  contributors  to  professional  journals  ...  in  the 
last  five  years,  have  written  some  fifty-four  articles 
and  reports  describing  and  evaluating  the  assessment 
center  method. 

McConnell  [45]  describes  the  record  of  a  one-day 

package  assessment  program: 

In  19.70,  at  the  end  of  a  five  year  research  and  develop- 
ment project,  the  American  Management  Association  (AMA) 
introduced  a  multimedia  assessment  centre  programme  .  .  . 
in  the  three  years  since  its  original  publication,  over 
150  organizations  have  used  it  for  assessment  of  their 
management  development  needs. 
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Byham  [11]  offered  this  estimate  of  assessment 

center  growth,  circa  1970: 

To  my  knowledge,  twenty  companies  have  been  responsible 
for  assessing  more  than  70,000  candidates  in  the  last 
ten  years,  but  at  least  100  more  companies  are  developing 
centers  or  are  in  an  advanced  stage  of  center  planning. 
Many  others  are  "looking  into  the  idea."  As  an  indication 
of  this  interest,  I  might  cite  the  fact  that  more  than 
200  company  representatives  attended  conferences  on  the 
assessment  center  method  during  1969. 

Two  articles  have  been  selected  to  provide  detail 

on  current  industrial  assessment  programs.   The  first  concerns 

a  one-day  "package  program"  offered  by  the  American  Management 

Association  (AMA) .   Such  a  program  puts  assessment  techniques 

within  reach  of  smaller  companies  who  cannot  afford  to  develop 

and  maintain  programs  of  their  own.   The  article  (by  McConnell 

[45])  describes  exportation  of  the  AMA  program  to  Europe  and 

is  written  with  a  British  flavor: 

The  assessment  centre  programme  consists  of  5  basic  parts: 

Chairman's  training- -A  representative  of  the  organization 
spends  one  week  at  an  intensive  training  programme  conduc- 
ted by  Management  Centre  Europe  to  obtain  all  of  the  know- 
ledge and  skills  necessary  to  conduct  all  phases  of  the 
programme  within  his  own  organization. 

Assessor  training- -Upon  returning  to  his  organization,  the 
trained  chairman  selects  five  existing  managers  to  be 
trained  as  evaluators  in  the  programme.   These  managers, 
once  trained,  are  referred  to  as  assessors.   Their  training 
consists  of  a  28  hour  multimedia  training  course  in  which 
they  are  taught  the. skills  and  techniques  necessary  to 
serve  successfully  as  assessors.   Included  in  this  training 
is  a  measuring  device  to  insure  their  skills  are  at  a  level 
at  which  they  can  successfully  serve  as  assessors. 

Management  simulation  workshop- -Once  the  assessors  are 
trained,  a  management  simulation  workshop  is  conducted. 
This  is  one  day  in  length  and  consists  of  eight  exercises. 
Twelve  participants  are  selected  to  go  through  the  eight 
exercises  and  are  observed  in  these  exercises  by  the 
assessors.   The  assessors'  role  in  the  workshop  is  to 
observe  and  record  the  behavior  of  the  participants.   Fol- 
lowing the  one  day  workshop,  the  participants  are  allowed 
to  return  to  their  jobs. 
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•Evaluation  meeting- -The  assessors  and  the  chairman  meet 
following  the  management  simulation  workshop  to  report 
and  evaluate  the  behavior  of  each  individual  observed. 
This  final  evaluation  yields  the  report  on  each  individual 
participant  in  12  specific  management  abilities,  overall 
management  ability,  and  the  individual's  organizational 
potential.   This  evaluation  meeting  requires  approximately 
12  hours. 

Feed-back- -At  a  feed-back  session  between  the  chairman 
and  each  participant,  the  results  of  the  centre  are  com- 
municated to  the  participant,  compared  with  his  own  self 
evaluation,  and  specific  development  plans  for  the  par- 
ticipant are  established. 

The  second  article  (Byham  [11])  describes  the  most 

widely  used  format:   the  five-day  program  patterned  after  the 

AT§T  model.   Although  the  following  material  was  based  on  the 

J.  C.  Penney  program,  it  is  generalized  enough  to  typify 

practices  of  many  of  the  major  firms. 

Sunday 

Six  management  assessors  meet  at  a  conveniently  located 
motel  and  organize  materials  for  the  week's  activities. 
Late  in  the  day,  twelve  candidates,  all  of  them  of  com- 
parable rank  in  the  company,  arrive  and  settle  in. 

Monday  Morning 

Period  1:   After  orientation  announcements  the  candidates 
are  divided  into  teams  of  four,  for  participation  in  a 
management  game.   Each  team  is  given  a  limited  amount  of 
capital  to  purchase  raw  materials,  make  a  product,  and 
sell  it.   The  raw  materials  are  usually  tinker-toy  parts 
which  can  be  assembled  into  a  variety  of  products  of  dif- 
ferent complexity,  each  of  which  has  a  different,  pre- 
specified  market  value. 

The  players  must  first  decide  how  to  invest  their 
capital  to  maximize  profits  and  then  organize  the  pur- 
chasing, manufacturing,  and  selling  operations.   Assessors 
observe  the  players  for  signs  of  leadership,  organizational 
ability,  financial  acumen,  quickness  of  thinking,  and 
efficiency  under  stress. 

Suddenly  the  players  are  notified  that  the  prices  of 
the  ra\v  materials  and  the  products  have  been  radically 
changed,  requiring  drastic  redeployment  of  capital  and 
extensive  operational  reorganization.   As  soon  as  they 
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have  regrouped,  these  prices  are  abruptly  changed  again. 
The  actions  the  players  take  allow  the  assessors  to  esti- 
mate their  adaptability. 

The  game  is  then  halted,  and  each  candidate  is  asked 
to  write  a  report  evaluating  his  own  performance  and  that 
of  his  fellow  players. 

Period  2:   The  candidates  are  divided  into  groups  of  six. 
While  one  group  takes  written  psychological  tests,  the 
members  of  the  other  group  are  interviewed  individually 
by  the  assessors.   The  assessors  have  been  provided  \vith 
detailed  background  information  on  each  man,  and  they  use 
this  to  probe  for  evidence  of  drive,  motivation,  and  sense 
of  self -development .   This  Assessment  Interview,  so  called, 
is  ordinarily  the  only  exercise  in  the  assessment  process 
that  focuses  on  the  candidate's  past  behavior. 

Monday  Afternoon 

Period  1:   The  testing  and  interviewing  groups  are  reversed, 

Period  2:   In  two  leaderless  groups  of  six,  the  candidates 
join  in  discussion  of  a  promotion  decision.   Here  the  can- 
didates play  the  role  of  supervisors  brought  together  on 
short  notice  by  their  boss  to  pick  one  man  from  a  pool  of 
six  for  advancement.   Each  candidate  receives  the  file  of 
one  of  the  men  in  the  pool,  whom  he  is  then  to  "champion1' 
for  the  promotion.   After  each  candidate  has  studied  his 
protege's  folder,  the  group  meets  for  an  hour's  discussion 
to  choose  the  man  it  will  recommend.   Assessors  observe 
the  candidates'  exchanges  in  the  meeting  for  signs  of 
aggressiveness,  persuasiveness,  expository  skill,  energy, 
flexibility,  self-confidence,  and  the  like. 

Alternative  exercise:   In  leaderless  groups  of  six,  candi- 
dates discuss  the  20  most  critical  functions  of  a  manager 
and  list  them  in  order  of  importance.   (This  forces  them 
to  think  about  the  qualities  on  which  they  are  being 
assessed.)   Each  group  then  chooses  a  spokesman  who  pre- 
sents the  list  and  the  rationale  behind  it  to  the  whole 
group  of  assessors  and  candidates. 

Monday  Evening 

Each  candidate  receives  material  on  how  best  to  conduct 
employment  interviews  and  also  the  resume  of  a  job  appli- 
cant.  He  studies  these  for  use  in  one  of  the  exercises 
on  the  following  day.   He  may  also  receive  special  phone 
calls--for  example,  the  Irate  Customer  Phone  Call. 
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Tuesday  Morning 

Tuesday  morning  is  devoted  to  the  In-Basket  Exercise.   This 
simulates  the   experience  a  candidate  would  have  if  he  were 
suddenly  and  unexpectedly  promoted  a  grade  or  two  and 
arrived  at  work  one  morning  to  find  his  in-basket  full  of 
unfamiliar  material  typical  of  the  sort  lie  would  then  have 
to.  handle.   Me  is  instructed  to  go  through  this  material 
and  deal  with  the  problems,  answer  the  inquiries,  request 
additional  information  where  he  needs  it,  delegate  tasks 
to  proper  subordinates,  and  generally  organize  and  plan 
just  as  he  would  if  he  had  actually  been  promoted. 

Tuesday  Afternoon 

Period  1:   The  candidates  conduct  the  employment  interviews 
for  which  they  prepared  the  night  before,  each  interview 
taking  place  in  the  presence  of  an  assessor.   The  appli- 
cants are  college  students  who  have  been  especially  trained 
in  the  applicant's  role.   The  interview  itself  lasts 
roughly  halfan  hour,  after  which  the  applicant  leaves  and 
the  assessor  quizzes  the  candidate  to  determine  what  in- 
sights he  has  obtained  about  the  applicant. 

Period  2:   The  next  exercise  is  the  resolution  of  disci- 
plinary cases.   In  groups  of  four,  candidates  decide  how 
to  allocate  their  time  between  three  such  cases  and  then 
decide  the  cases  themselves,  within  one  hour.   This  exer- 
cise provides  the  assessors  with  information  on  a  candi- 
date's appreciation  of  personnel  problems  and  his 
sensitivity  to  subordinates'  views  of  events  and  actions, 
as  well  as  insight  into  his  behavior  within  a  group. 

Alternative  exercise:   The  candidates  are  assigned  roles 
as  city  councilmen  \\?ho  meet  to  allocate  a  $1  million 
federal  grant  to  the  city  departments.   Each  "councilman" 
interprets  a  briefing  document   provided  by  a  city  agency- - 
the  police  department,  sanitation  department,  water  depart- 
ment, and  so  on- -and  tries  to  get  as  much  of  the  grant 
allocated  to  this  agency  as  possible.   Again,  effective 
discussion  is  limited  to  one  hour. 

Tuesday  Evening 

Detailed  data  on  a  company  was  provided  to  all  the  candi- 
dates.  Each  is  asked  to  examine  its  financial  and 
marketing  situation  from  the  viewpoint  of  a  consultant 
and  to  prepare  a  written  recommendation  for  its  board  of 
directors  on  the  future  expansion  of  a  particular  part 
of  its  product  line. 

At  the  same  time,  also  in  preparation  for  the  next 
day's  activities,  the  assessors  study  the  results  of  the 
candidates'  In-Basket  tests  in  detail. 
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Wednesday 

Period  1:   Four  groups  are  formed,  each  consisting  of  three 
candidates  and  an  assessor.   Each  candidate  takes  his  turn 
presenting  his  oral  analysis  of  the  company  data  studied 
the  night  before  and  submitting  written  recommendations. 

Period  2:   These  three  candidates  work  together  for  an 
hour  to  reconcile  and  consolidate  the  recommendations. 

Period  3:   The  In-Basket  Interview  follows,  in  which  an 
assessor  discusses  with  a  candidate  the  various  actions 
he  took.   This  further  defines  each  man's  grasp  of  typical 
problems  and  opportunities. 

Wednesday  Afternoon 

In  a  final  group  session,  the  candidates  rate  each  other 
and  ask  any. questions  they  may  have.   They  then  leave  for 
home . 

Wednesday  Afternoon  to  Friday 

The  assessors  discuss  the  candidates  and  prepare  their 
ratings  and  reports. 

Throughout  all  the  exercises,  the  assessors  have  been 
rotated  so  that  as  many  as  possible  have  had  a  chance  tc 
observe  each  candidate  closely.   Thus,  in  these  discussions, 
the  assessor  who  conducted  Jones's  personal  interview 
summarizes  his  background  and  his  own  impressions  of  his 
behavior  in  the  interview;  next  the  assessor  who  checked 
what  Jones  did  in  his  In-Basket  Exercise  and  interviewed 
him  on  it  presents  his  impressions;  and  so  on.   Each 
assessor  attempts  to  keep  these  descriptions  nonevaluative 
and  objective. 

Only  when  all  the  assessors  who  have  observed  Jones 
have  spoken  does  the  group  begin  to  judge  his  behavior 
from  the  viewpoint  of  his  management  potential  and  the 
directions  in  which  he  needs  to  develop.   After  they  have 
reached  a  consensus,  they  prepare  a  final  report. 

Within  two  weeks  a  manager  who  has  had  experience  as 
an  assessor  meets  with  Jones  to  communicate  the  results. 
In  this  meeting  he  lays  stress  on  the  areas  in  which  Jones 
needs  to  develop  himself  and  encourages  him  to  set  appro- 
priate goals. 
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b.   Military 

Four  sources  have  been  selected  to  provide  samples 
of  widely  divergent  military  programs.   The  first,  Snyder  [124] 
describes  Inventory  and  Assessment  at  the  United  States  Army 
War  College  (AWC) .   Although  this  program  is  atypical  of  the 
assessment  programs  described  thus  far,  it  is  of  interest 
because  it  is  purely  development-oriented.   Instead  of  a  tightly 
scheduled  program  of  a  few  days  duration,  the  assessment  is 
interspersed  throughout  the  academic  year  (September  -  June) . 
Another  difference  is  that  faculty  coaches  or  counselors  replace 
the  assessors  previously  described.   Each  coach  is  assigned  15 
students  to  counsel  throughout  the  school  year.   The  program 
employs  five  assessment  techniques,  e.g.,  Baseline,  Inventory, 
Optional  Personal  Inventory,  Professional  Self -Assessment, 
Experiential  Assessment, and  Value-Gained  Inventory.   They  are 
described  as  follows: 

(1)   Baseline  Inventory.   A  questionnaire  is  mailed 
to  the  student  prior  to  his  arrival  at  the  AWC.   The  student 
indicates  his  competence  in  a  variety  of  areas  related  to  the 
curriculum  and  returns  the  questionnaire,  whereupon  the  results 
are  compiled.   Feedback  is  provided  which  shows  the  class  range 
and  mean  on  each  item.   In  this  way,  the  student  and  the  school 
can  tailor  individual  curricula  to  best  advantage.   Future 
plans  are  to  include  additional  instruments  in  this  phase,  such 
as:   the  Graduate  Record  Exam,  to  assess  academic  potential, 
and  other  tests  for  reading  comprehension,  listening  compre- 
hension, writing  skills,  etc. 
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(2)  Optional  Personal  Inventory  (0P1).   Research 
on  executive  development  is  said   to  identify  a  need  for 
reassessment  of  personal  and  professional  goals  which  occurs 
when  an  individual  is  about  age  forty  and  two-thirds  of  the 
way  through  his  career.   The  OPI  was  designed  to  meet  this 
need.   A  battery  of  tests  is  available  to  the  student  which 
can  be  administered  and  analyzed  by  his  faculty  coach.   The 
battery  includes  measures  of  speaking,  reading  and  writing 
abilities,  attitudes,  personality  traits,  needs,  vocational 
interests ,  etc. . 

(3)  Professional  Self -Assessment .   This  item 
provides  feedback  in  three  ways:   first,  to  the  student,  second 
to  the  instructor,  and  third  to  the  institution.   The  need  for 
such  a  feedback  system  is  underscored  by  the  fact  that  the 

AWC  does  not  utilize  an  academic  grading  system.   The  self- 
assessment  consists  of  a  final  examination  administered  to  the 
students  after  which  they  are  provided  with  solutions  to  enable 
self -grading .   The  unsigned  exam  papers  are  then  given  to  the 
instructor  for  analysis  by  him  and  the  institution. 

(4)  Experiential  Assessment.   A  group  problem- 
solving  exercise  is  conducted  by  a  team  of  students  lead  by 
a  faculty  coach.   Upon  completion,  exercise  performance  is 
critiqued  by  a  panel  of  experts.   Each  student  is  provided 
feedback  on  his  performance  by  the  faculty  team-leader,  his 
peers  and  the  panel  of  experts. 
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(5)   Value-Gained  Inventory.   Essentially,  admin- 
istration of  alternate,  but  equivalent,  forms  of  the  Baseline 
Inventory  at  the  end  of  the  academic  year.   Statistical  analysis 
would  be  utilized  to  determine  statistically  significant  changes. 
Results  could  then  be  provided  to  students  and  faculty,  and 
also  used  in  curriculum  development. 

The  AWC  program  was  initiated  in  academic  year 
1973  -  1974,  therefore  it  is  too  early  for  an  evaluation,  how- 
ever,- the  implications  for  the  Naval  Postgraduate  School  (NPS) 
are  obvious  and  would  suggest  a  close  and  continuing  follow-up. 

The  second  military  application  to  be  considered 
is  the  pilot  assessment  center  project  at  the  United  States 
Army  Infantry  School  (USAIS) ,  Fort  Benning,  as  reported  by 
Veaudry  and  Campbell  [65]  .   This  program  is  patterned  after 
the  "AT§T  model"  and  entails  three  days  of  intensive  assess- 
ment.  There  are  actually  three  separate  programs:   one  for 
senior  noncommissioned  officers  (NCOs) ,  a  second  for  lieutenants, 
and  a  third  for  captains,  however,  they  follow  the  same  basic 
philosophy  and  share  many  of  the  commercial  instruments,  e.g., 
paper  and  pencil  tests,  inventories,  etc..   The  programs  are 
based  on  the  "Leadership  for  Professionals"  study  conducted 
by  the  Army  War  College.   Ten  exercises  were  designed  for 
each  program,  and  combined  with  batteries  of  paper  and  pencil 
tests.   The  resulting  assessments  include  writing  exercises, 
in-baskets,  leaderless  group  discussions,  war  games,  field 
exercises,  simulations  of  organizational  problems  in  the 
field  and  three  types  of  interviews. 
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Evaluation  reports  consist  of  numerical  ratings 
and  narrative  profiles  describing  candidates'  strengths  and 
weaknesses.   The  narrative  profiles  include  a  one  page  summary 
of  abilities  and  an  eight  to  twelve  page  detailed  description 
based  on  the  observations  of  the  assessors. 

The  pilot  assessment  center  project  was  chartered 

for  one  year  and  given  the  following  missions: 

Test  the  assessment  concept  as  a  leadership  and  career 
development  technique. 

Test  the  assessment  concept  as  a  precommission  screening 
device. 

Determine  the  feasibility  and  applicability  for  Armywide 
use  of  the  concept  and  techniques  developed  under  the  first 
two  steps. 

As  with  the  AWC  program,  the  assessment  center  project  has 
significance  for  the  Navy  and  warrants  close  scrutiny. 

The  British  Royal  Commissions  Board  (RGB)  is  one 
of  the  oldest  military  assessment  programs  in  continuous  oper- 
ation.  The  USAIS  Assessment  Center  Project  trip  report  [126] 
provides  the  following  overview  of  the  RCB : 

The  RCB  .  .  .  has  the  mission  of  selecting  candidates 
with  the  potential  qualities  of  character,  ability  and  leader 
ship  necessary  for  a  career  as  an  officer.   All  candidates 
for  either  Short  Service,  Limited  Service  or  Regular  Com- 
missions must  be  evaluated  by  the  RCB  prior  to  any  pre- 
commissioning  training  (Sandhurst  or  University  course). 

The  RCB  has  a  President  (Major  General),  three  Vice 
Presidents  (Brigadiers),  a  Training  Officer  (LTC) ,  (and 
twelve  assessors)  (six  Lieutenant  Colonels  and  six  Majors) , 
all  assigned  on  two  year  tours.    The  Board  has  the  capa- 
bility of  assessing  48  candidates  each  week,  organized 
into  6  groups  of  8  each.   Each  group,  therefore,  is  directly 
supervised  by  a  Major,  whose  principal  function  is  detailed 
observing,  and  a  LTC,  whose  functions  include  observing  and 
conducting  in-depth  interviews  of  each  of  the  8  candidates. 

Although  there  is  no  permanent  psychologist  assigned 
to  the  Board,  the  Board  draws  on  the  professional  advice 
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of  psychologists  assigned  to  the  Army  Personnel  Service, 
and  a  civilian  firm  of  consultants.   In  addition,  each 
group  of  candidates  receives  an  educational/intelligence 
rating  from  an  Education  Advisor  attached  to  the  RCB  from 
the  Army  Education  Board. 

The  criteria  used  for  evaluating  each  candidate  are 
(1)  Training  Potential,  (2)  Applied  Ability,  and  (3) 
Character.   Within  each  major  criterion,  the  following 
facets  are  considered: 


(1)  Training  Potential 


(IQ) 


(a)  Officer  intelligence  rating 

(b)  Educational  standard 

(c)  Self-expression  (written  and  oral) . 

The  Training  Potential  is  assessed  by  the  Education 
Advisor,  based  on  records  (achievement  vs.  opportunities) 
and  tests  conducted  at  the  RCB.   The  candidate's  overall 
rating  is  given  a  numerical  value  as  follows: 


Strong 


Good 


Adequate 


Limited 


Weak 


10,9        8         7 

(2)  Applied  Ability 

(a)  Planning  ability 

(b)  Practical  ability 

(c)  Physical  ability. 

(3)  Character 


5,4,3,2,1 


(a 
(b 
(c 
(d 
(e 
(f 
(g 
(h 
(i 
(J 
(k 


Coolness  (ability  to  relax) 

Initiative 

Sense  of  urgency 

Determination 

Military  compatibility 

Sense  of  responsibility 

Sense  of  awareness  (breadth  of  knowledge) 

Liveliness 

Dominance 

Quality  of  character 

Range  of  personal  relations  (ability  to  mix) . 


The  schedule  for  each  session  is  divided  into  Parts  I 
and  II.   Due  to  the  large  number  of  candidates,  it  was  felt 
necessary  to  establish  an  initial  screening  process  to 
reduce  the  number  of  candidates  down  to  48  for  the  actual 
assessment.   The  schedules  are  as  follows: 
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PART  I 

Monday  Tuesday 

Intelligence  test  Part  I  Board  sits 

Current  affairs  Eliminated  depart 

Essay 

Interviews 

Evaluation  of  reports 

PART  II 


Tuesday  Wednesday  Thursday 

Group  discussion  Written  projects  Group  race 

Group  task  Individual  tasks  Final  Boards 

Group  race  Obstacle  course  Results  posted 

Interviews  Lecturettes 

The  Majors  and  LTC's  are  rotated  each  session  to  prevent 
bias,  and  their  recording  of  information  is  done  independently, 
The  Major  simply  observes;  he  does  no  interviewing,  neither 
does  he  see  any  written  reports  or  references  on  the  candi- 
dates.  In  addition  to  observing^  the  LTC  conducts  interviews 
in  depth,  but  also  does  not  see  any  reports  or  references. 
A  preliminary  finding  is  determined  after  the  first  day  with 
the  Brigadier  reconciling  the  differences  between  the  Major 
and  the  LTC.   The  reports  and  references  have  been  previously 
studied  and  graded  by  the  Brigadier. 

Based  on  the  preliminary  findings  (Straight  up,  Doubtful 
up,  Doubtful,  Doubtful  down,  Straight  down),  the  President 
of  the  Board  will  then  conduct  his  own  interviews,  observe 
candidates  as  they  (perform)  executive  leadership  "tasks," 
and  attend  those  final  boards  he  feels  are  critical.   The 
Brigadier  makes  the  final,  overall  assessment  rating, 
subject  to  the  approval  of  the  President.   The  rating  is 
either  Pass,  Fail  or  Return;  with  copies  forwarded  to 
Sandhurst  and  the  Ministry  of  Defense.   Historically,  the 
Board's  pass  rate  has  been  some  19  percent  for  Sandhurst 
(Regular  Commission)  candidates,  11  percent  for  Limited 
Service  candidates  and  15  percent  overall. 

The  final  military  assessment  program  to  be  des- 
cribed is  the  recently  discontinued  Staff  Officer  and  Selection 
Course  of  the  Federal  German  Navy  described  by  Schmahling  [57]. 
This  six-week  assessment  program  was  in  existence  from  February, 
1959  to  December,  1973.   During  its  fifteen  years  of  operation, 
2287  officers  underwent  assessment. 
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The  basic  objective  of  the  program  was  selection 
of  Staff  Officers.   In  1966,  the  selection  category  of  Admiral 
Staff  Officer  was  added.   The  criteria  used  consisted  of  three 
sets  of  variables  pertaining  to  personality,  mental  ability  and 
problem-solving  capability.   It  became  evident  that  personality 
traits  were  poor  predictors  and  the  program  was  finally  based 
entirely  on  assessment  of  mental  abilities. 

As  of  1968,  an  overall  score  has  been  obtained  by 
combining  scores  from  eight  items,  each  with  equal  weight. 
These  eight  items  include:   four  written  tests,  three  essay- 
type  exams,  a  term  paper,  four  oral  tests,  two  in-basket  simu- 
lations, a  war  game  and  an  oral  presentation.   During  the  years 
the  program  underwent  constant  refinement,  and  efforts  were 
made  to  control  for  bias  due  to  the  differing   backgrounds 
of  the  candidates.   Tests  were  selected  and  designed  to  afford 
responses  from  different  levels  of  candidate  experience  and 
sophistication.   Emphasis  was  on  methodology  vice  recall  of 
previously  learned  knowledge. 

Despite  the  fact  that  the  program  was  selection- 
oriented,  a  number  of  developmental  effects  were  noted.   For 
most  officers,  the  return  to  academic  pursuits  after  years  of 
operational  service  was  a  challenge.   Another  stimulus  was 
provided  by  the  high  standard  of  performance  demanded  and 
obtained  which  produced  a  wholesome  competition  between  candi- 
dates.  A  third  factor  was  the  knowledge  that  assessors  were 
ever-present  and  observing  every  nuance  of  the  candidates' 
performance . 
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The  only  requirement  cited  for  assessors  was  that 
one  assessor  for  each  group  of  ten  candidates  had  to  have  com- 
pleted Admiral's  Staff  training.   Thus,  assessors  typically 
possessed  broad  operational  experience  and  were  expected  to 
have  a  sense  of  responsibility  for  their  fellow  officers  and 
the  capability  of  making  pragmatic  decisions.   The  practice  of 
multiple  assessment  was  seen  as  easing  the  burden  of  responsi- 
bility for  the  assessor  and  increasing  the  candidate's  confi- 
dence in  the  final  rating. 

With  few  exceptions,  the  candidate  received  only 
the  final  result  upon  completion  of  the  course.   Two  reasons 
given  for  this  practice  were  to  save  time  and  to  prevent 
weak  candidates  from  giving  up  midway  through  the  program. 
(This  writer  does  not  consider  either  of  these  very  compelling 
and  speculates  that  the  organization  had  other  reasons  for 
withholding  feedback) .   Schmahling  asserts  that  the  lack  of 
feedback  was  rarely  criticized  but  that  many  candidates  felt 
that  past  performance  evaluations  should  have  been  considered 
in  the  final  rating. 

The  Federal  German  Navy  program  has  been  replaced 
by  a  tri-service  assessment  center,  however,  as  the  author 
points  out,  the  fifteen  years  of  assessment  experience  is  not 
lost,  but  can  be  profitably  applied  to  the  new  assessment 
program. 

3.   Advantages  (Support  for  the  concept) 

Three  articles  have  been  selected  which  state  the 
most  commonly  cited  benefits  of  assessment  centers.   The  first 
is  by  Ginsburg  and  Silverman  [25]  who  described  the  design  and 
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implementation  of  an  assessment  program  for  hospital  personnel. 

They  wrote  as  follows: 

Compared  to  other  forms  of  personnel  appraisal,  this  method 
is  seen  to  be  more  effective  because  all  assessees:   (1) 
have  an  equal  opportunity  to  display  their  talents   (2) 
are  seen  under  similar  conditions  in  relevant  situations 
designed  to  bring  out  the  particular  skills  and  abilities 
needed  for  the  position  or  positions  for  which  they  are 
being  considered,  and  (3)  are  evaluated  by  a  team  of 
trained  assessors,  unbiased  by  past  association,  who  are 
intimately  familiar  with  both  the  position  requirements 
and  the  institutional  climate  ... 

The  major  contribution  of  the  multiple  assessment 
approach  has  been  the  use  of  situational  tests  or  exercises 
.  .  .  .  the  application  of  situational  techniques  to 
assessment  has  reduced  the  amount  of  inferences  which  must 
be  made  from  the  more  loosely  structured  paper  and  pencil 
techniques.   They  provide  more  positive  answers  to  the 
question,  "Given  these  traits,  how  is  he  likely  to  behave 
in  a  work  situation?" 

Situational  methods  also  offer  the  potential  of  adding 
greatly  to  the  scope  of  human  characteristics  which  can  be 
evaluated.   Although  more  expensive  and  time-consuming  to 
administer  than  usual  appraisal  procedures,  the  need  to 
find  ways  of  evaluating  characteristics  not  covered  by  the 
latter  is  sufficient  to  warrant  extensive  experimentation 
with  relatively  elaborate  techniques  .... 

All  in  all,  the  center  does  seem  to  be  predictive  of 
managerial  ability.   Furthermore,  a  vital  part  of  the 
Identification  and  Development  Center  is  the  action  which 
will  be  taken  to  increase  individual  effectiveness.   Tailored 
developmental  programs  will  replace  shot-gun  attempts  at 
training  individuals  for  increased  responsibility. 

Expanding  on  the  benefits  of  development,  Slevin  [59] 

wrote: 

The  assessment  center  can  serve  as  a  management  develop- 
ment tool  in  two  ways.   First,  the  exercise  of  participating 
in  a  center  is  in  itself  a  developmental  experience.   The 
individual  gets  practice  in  leadership  skills,  makes  oral 
presentations,  makes  hypothetical  management  decisions  and 
has  them  criticized,  observes  himself  on  a  video  tape 
replay  of  his  performance  and  has  other  developmental 
experiences.   Research  has  indicated  that  the  video  tape 
feedback  alone  has  very  promising  potential  for  organizational 
development  team  building  and  other  efforts  (Weber  [66]). 
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In  addition,  the  whole  process  of  the  critical  and  intensive 
period  of  self  insight  that  is  experienced  in  a  two  to  three 
day  center  is  extremely  constructive  in  providing  the 
assessee  with  information  on  how  to  achieve  positive  change. 
He  is  provided  with  an  opportunity  to  scrutinize  his  behavior 
in  a  critical  manner  not  possible  in  on-the-job  performance, 
and  in  a  manner  that  might  lead  him  to  understand  those 
behaviors  that  are  most  effective  and  least  effective  in 
his  managerial  capacity. 

Second,  the  assessment  center  is  an  excellent  place  for 
the  formulation  of  a  developmental  plan.   Executives  serving 
as  assessors  are  currently  operating  at  levels  in  the  organ- 
ization to  which  the  assessee  eventually  aspires.   They're 
in  a  position  to  evaluate  the  types  of  experience  and 
training  the  assessee  needs  to  reach  their  level  of  attain- 
ment.  At  no  other  time  in  the  assessee' s  experience  with 
his  organization  will  six  upper  management  executives  spend 
an  intensive  two  to  three  hours  discussing  his  strengths, 
weaknesses  and  developmental  needs.   At  this  point,  in 
conjunction  with  the  overall  manpower  needs  of  the  organi- 
zation, a  developmental  plan  may  be  engineered  that  is 
specific  to  the  needs  of  this  assessee.   This  makes  obsolete 
the  conception  of  a  standardized  developmental  approach  for 
all  management  and  provides  the  unique  capacity  of  a  specif- 
ically engineered  developmental  program  for  each  person 
assessed. 

The  third  article  is  by  Byham  [11]  who  summarized 

assessment  center  advantages  as  follows: 

Reports  have  proved  to  be  remarkably  valid.   Longitudinal 
studies  of  thousands  of  employees  assessed  over  the  last  few 
years  indicate  that  this  assessment  method  is  much  more 
accurate  than  traditional  appraisal  procedures,  and  these 
seem  to  be  the  reasons: 

*The  exercises  used  are  designed  to  bring  out  the  specific 
skills  and  aptitudes  needed  in  the  position(s)  for  which 
a  group  of  candidates  is  being  assessed. 

(Note:   This  facet  is  commonly  referred  to  as  "combating 
the  Peter  Principle;  since  the  candidate  is  evaluated  on 
requirements  of  the  position   he  is  aspiring  to- -not  those  he 
he  has  held  in  the  past.   When  there  is  a  large  difference 
between  these  requirements- -e . g . ,  promotion  from  crafts  to  first 
line  management- -this  factor  becomes  very  significant). 

*Since  the  exercises  are  standardized, assessors  evaluate 
the  candidates  under  relatively  constant  conditions  and 
thus  are  able  to  make  valid  comparative  judgments. 
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*The  assessors  usually  do  not  know  the  candidates  personally 
so,  being  emotionally  disengaged,  they  are  unbiased. 

*The  assessors  are  shielded  from  the  many  interruptions  of 
normal  working  conditions  and  can  pay  full  attention  to  the 
candidates'  behavior  in  the  exercises. 

*The  procedures  focus  their  attention  on  the  primary  kinds 
of  behavior  they  ought  to  observe  in  evaluating  a  promotion 
candidate . 

*They  have  been  trained  to  observe  and  evaluate  these  kinds 
of  behavior. 

Byham  further  points  out  some  indirect  benefits  which 

accrue  from  the  center,  e.g.: 

a.  Candidate  training 

b.  Positive  influence  on  morale  and  job  expectations 

c.  Subtle  improvement  of  candidates'  understanding 
of  and  attitudes  toward  organizational  goals  and 
policies 

d.  Assessor  training  (said  to  be  by  far  the  most 
valuable  "fringe  benefit") . 

As  a  final  argument,   Byham  tallies  his  survey  of 

users  of  the  centers: 

In  a  survey  of  the  20  companies  that  operated  centers, 
I  uncovered  some  22  studies  in  all  that  showed  assessment 
more  effective  than  other  approaches  and  only  one  that 
showed  it  exactly  as  effective  as  some  other  approaches.   None 
showed  it  less  effective.   As  I  suggested  before,  these 
studies  exhibit  correlations  between  center  predictions 
and  achievement  criteria  such  as  advancement,  salary  grade, 
and  performance  ratings  that  range  as  high  as  .64.   The 
companies  appear  satisfied  that  they  are  on  the  right  track. 

Byham  and  Pentecost  [12]  echo  most  of  the  preceding 

with  the  following  additional  observations: 

a.  ...  participation  in  an  assessment  center  stimulates 
the  candidates  self -development  through  self -insight . 

b.  Appropriate  training  and  development  programs  can  be 
organized  and,  if  practical,  jobs  can  be  developed  to  match 
the  particular  abilities  of  available  candidates. 
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The  report  of  a  seminar  conducted  by  the  Foundation 

for  Research  on  Human  Behavior  [112]  illustrates  the  potential 

impact  of  face  validity  on  group  morale  : 

It  is  important  that  the  promotion  of  a  given  individual 
be  not  only  the  best  selection  but  also  that  it  appear  to 
other  candidates  as  a  good  selection.   If  a  candidate  is 
judged  highly  qualified  mainly  on  the  basis  of  tests,  con- 
sultants' opinions,  and  other  information  relatively  invisible 
to  the  candidate  population,  the  advantages  of  promoting  him 
have  to  be  balanced  against  the  possible  disadvantages  to  the 
morale  of  the  group.   (However)  if  the  candidates  consider 
that  the  evaluation  standards  are  clearly  relevant  to  the 
job  situation,  they  will  tend  to  accept  the  results  of  the 
/  evaluations. 

Kraut  and  Scott  [41]  continue  this  theme: 

But  face  validity  ...  is  high  for  assessment  programs. 
Raters  in  an  assessment  program  generally  come  away  feeling 
that  it  does  indeed  work.   The  exercises  seem  reasonable, 
and  one  can  make  discriminations  which  appear  meaningful. 

They  also  report  results  of  a  study  which  they  conducted 
involving  1086  subjects.   The  results  showed  that  assessment 
center  ratings  correlated  highly  with  two  major  criteria: 
(1)  promotion  to  the  second  level  of  management  and  (2)  demo- 
tions from  the  first  level  of  management.   Both  of  the  criteria 
were  chosen  to  minimize  contamination.   Since  the  assessment 
objective  was  selection  for  promotion  to  first  level  management, 
the  authors  contended  that  the  criteria  selected  were  minimally 
contaminated  by  knowledge  of  assessment  center  ratings. 

Kraut  [40]  cites  studies  by  Bray  and  Grant  [10],  Wollowick 
and  McNamara  [71]  and  Kraut  and  Scott  [41]  .   In  summary  he 
states : 

These  studies  and  many  others  lead  one  to  conclude  that 
assessment  programs  have  validity  in  predicting  those  who 
will  move  ahead  in  an  organization.  Many  of  the  studies  have 
flaws,  but  there  is  a  consistent  pattern  of  apparent  validity. 
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He  also  discusses  a  phenomenon  reported  by  many  studies: 

that  the  predictive  validity  of  assessment  center  ratings 

improves  as  candidates  are  considered  for  promotions  to  second 

and  third  levels  of  management.   He  speculates  on  the  reason 

for  this : 

It  may  be  that  the  performance  of  first-line  managers 
in  this  company  is  heavily  dependent  on  technical  skills 
which  the  men  carry  into  that  level  of  management  (first) 
and  that  the  power  of  the  assessment  center  shows  up  pri- 
marily when  they  are  considered  for  higher  levels  of 
management . 

In  reviewing  a  previous  study  by  Hinrichs  [32] ,  Kraut 
points  out  a  possible  key  to  the  advantage  of  assessment  centers 
over  traditional  techniques.   He  says  that  observed  interpersonal 
activity  was  a  major  contribution  in  the  assessment,  but  not 
in  traditional  selection  procedures. 

In  response  to  criticisms  of  "deciding  a  man's  career 
on  two  days  of  observation,"  he  writes:   "The  assessment  center 
may  represent  two  days  more  of  observation  than  existed  before." 
Of  course  the  original  criticism  is  fallacious  since  most  assess- 
ment results  augment  rather  than  supplant  performance  appraisals 
and  other  sources  of  information  used  in  promotion  decisions. 

Dodd  [18]  reports  a  study  from  which  he  draws  an 

interesting  conclusion  regarding  assessment  techniques  and 

stereotyping: 

The  present  study  offers  some  hope  that,  far  from  sup- 
porting conformity  or  suppressing  independence,  tests  vali- 
dated against  assessment  performance,  when  used  to  supplement 
supervisory  recognition,  could  break  the  hold  of  conformity 
on  present  selection  procedures  and  open  management  ranks 
to  those  who  value  independence. 

The  undisputed  landmark  in  the  field  of  assessment 

center  validation  studies  is  the  Management  Progress    udy 
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of  AT^T.   No  article  on  the  subject  can  be  considered  inclusive 
without  paying  homage  to  "The  Study."   Bray  and  Grant  [10]  offer 
the  most  definitive  article  on  this  study  and  it  is  augmented 
by  a  series  of  articles  amplifying  on  it  and  providing  details 
of  other  AT§T  validation  studies. 

Bray  and  Campbell  [9]  describe  a  study  on  the  selection 
of  salesmen  which  yielded  a  predictive  validity  of  .51  for 
assessment  center  ratings  correlated  to  job  performance  ratings 
obtained  by  special  evaluation  teams. 

Campbell  and  Bray  [14]  discuss  criterion  and  contami- 
nation problems  in  validity  studies.   They  cite  two  studies 
which  attempted  to  overcome  these.   One  was  a  Michigan  Bell 
study  which  compared  the  first  forty  men  assessed  and  promoted 
with  the  last  forty  men  promoted  before  the  assessment  program 
began.   The  other  study,  done  in  the  New  England  Company,  compared 
candidates  rated  "acceptable"  with  thoserated  "not  acceptable." 
In  both  of  these  studies  the  ratings  of  the  assessment  center 
correlated  highly  with  subsequent  performance  and  were  superior 
to  traditional  selection  procedures.   The  authors  make  a  final 
important  point  that  predictive  validity  alone  is  not  sufficient 
but  that  acceptance  by  line  management  is  also  needed  if  a 
program  is  to  be  successful. 

Although  Cohen  and  Jaffee  [16]  don't  offer  any  advan- 
tages not  previously  mentioned,  they  do  make  three  predictions 
for  the  future  of  assessment  centers  in  government  applications: 
(1)  that  development  will  become  a  more  important  objective 
than  selection  or  internal  promotion;  (2)  that  future  programs 
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will  utilize  a  greater  number  of  new  exercises  simulating 

management  situations  in  government,  and  (3)  that  the  number 

of  assessment  centers  in  government  will  increase,  as  will 

interest  in  them. 

Miller  [50]  discusses  the  nature  of  potential  and 

intimates  that  a  single  job  may  be  an  inadequate  measure  of 

a  man's  full  potential: 

The  so-called  potential  of  an  individual  is  not  a  single 
ability,  but  a  combination  of  abilities  to  perform  effectively 
on  a  number  of  specific  jobs. 

Slevin  [59]  writes  to  the  same  problem  and  points  out  the 

inadequacy  of  traditional  personnel  procedures : 

Thousands  of  psychological  tests  have  been  developed 
for  the  specific  evaluation  of  personality  traits  and  mental 
aptitudes,  but  they  tend  to  be  rifle  shot  approaches  that 
evaluate  some  narrow  aspect  of  executive  potential. 

He  goes  on  to  point  out  that: 

In  contrast,  the  assessment  center  provides  a"broad  band" 
approach  to  the  evaluation  of  executive  potential. 

Another  point  mentioned  frequently  in  the  literature 

is  the  effect  of  job  assignment  on  an  individual's  career. 

Miller  [50]  describes  it  thus: 

The  company  is  apt  to  place  its  high  potential  personnel 
in  more  challenging  jobs,  whereas  low  or  no-potential  per- 
sonnel will  be  left  where  they  are  or  reassigned  to  less 
responsible  positions.   Thus  they  are  offered  little  or  no 
opportunity  to  display  their  qualifications  for  promotion. 

Thus,  if  such  assignment  is  based  on  valid  selection  procedures, 

a  good  man  may  be  placed  in  a  dead-end  job  and  left  to  stagnate. 

Miller  refers  to  this  hopeless  situation  as  the  "Darwinian 

theory  of  natural  selection,"  and  points  out  the  hope 

offered  by  assessment  techniques.   Since  all  candidates  compete 

on  an  equal  basis  regardless  of  current  assignment,  the  "have-nots" 

are  not  handicapped. 
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Pomerleau  [53]  makes  this  same  point,  but  carries  it 

further.   He  points  out  the  "image  effect"  whereby  managers  tend 

to  reward  subordinates  who  act  like  them.   This  can  utlimately 

lead  to  disaster,  since: 

Selecting  an  individual  primarily  because  he  has  no 
apparent  weaknesses  or  because  his  background  and  attitudes 
correspond  to  incumbent  executives  often  leads  to  the 
selection  of  mediocrities  who  have  no  perceptible  strengths. 

(For  a  more  detailed  treatment  of  this  topic,  see  Senger  [58]). 

Pomerleau  concludes  that  the  assessment  center  is  the  solution 

to  biased  supervisors'  ratings  since  it: 

.  .  .  ensures  that  the  individual  is  observed  and  evaluated 
by  a  team  of  trained  evaluators  unbiased  by  previous  associ- 
ations, who  are  familiar  with  the  skills  to  be  measured, 
the  personal  characteristics  desired,  and  the  environment 
in  which  the  work  is  performed. 

A  major  advantage  of  assessment  centers  is  the  relative 

freedom  from  charges  of  bias  in  selection  practices.   Pomerleau 

[53]  expresses  the  situation  thus: 

Insofar  as  the  assessment  center  technique  is  strictly 
job-related,  employing  exercises  which  seek  to  maximize 
objectivity,  and  further,  since  the  concept  has  never  been 
challenged  by  EEO  (Equal  Employment  Opportunity)  advocates, 
promoters  of  the  concept  may  be  encouraged. 

This  point  is  amplified  in  an  article  appearing  in  the 

August  28,  1971  issue  of  Business  Week: 

Dr.  Cabot  Jaffee  thinks  that,  since  the  assessment 
method  is  a  job  sample,  it  may  be  a  way  to  avoid  charges 
of  discrimination  that  sometimes  crop  up  over  paper  and 
pencil  tests.   He  cites  a  Labor  Department  project 
where  he  put  a  group  of  black  and  Indian  trainees  through 
conventional  written  tests  and  then  through  an  assessment 
center.   The  conventional  tests  showed  promotional  potential 
in  only  151  of  them.   Assessment  found  it  in  50%  of  them. 
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Because  of  this  built-in  "anti-bias"  factor,  the  assess- 
ment technique  may  get  a  boost  from  a  recent  Supreme  Court 
ruling.   Last  March,  the  court  ruled  that,  when  a  company 
uses  personnel  tests  or  degree  requirements  to  screen  candi- 
dates for  a  j ob  or  for  a  promotion,  it  must  be  able  to 
show  the  connections  between  its  screening  method  and  the 
job.   The  court  should  be  satisfied  with  assessment,  since 
it's  based  entirely  on  the  job. 

Jaffee,  et.  al.  [36]  address  this  subject  in  depth 
as  does  Dunnette  [109] .   A  slightly  different  aspect  of  this 
general  topic  is  covered  by  Bray  [8]  who  writes  on  opportunities 
for  women.   He  concludes  that:   "The  assessment  center  can  go 
a  long  way  .  .  .  toward  opening  up  greater  opportunities  for 
the  female  employee  and  bringing  about  fuller  utilization  in 
the  shortest  possible  time." 

Despite  the  sources  previously  cited,  there  are  some 

caveats  in  the  field  of  EEO  which  must  be  considered.   Bray  and 

Moses  [76]  ,  in  a  review  of  the  literature  from  1968  through 

1970  offer  these  findings  and  conclusions: 

.  .  .  considerable  attention  was  devoted  to  developing 
procedures  which  would  result  in  similar  test  performance 
among  racially  mixed  groups.   Generally,  none  of  these 
approaches  have  proved  to  be  successful.   The  attempts  to 
assess  and  compensate  for  bias  in  testing  procedures  have 
considered  the  following  aspects:   developing  "culture -fair" 
tests;  using  nonverbal  tests;  providing  special  test 
conditions;  providing  special  test  training  .  .  . 

The  culture-fair  fad  appears  over.  One  factor  hastening 
its  demise  was  the  finding  that  nonverbal  tests  did  not 
create  additional  fairness  for  disadvantaged  group  members. 
Nonverbal  tests  were  initially  viewed  as  approximations 
of  culture-fair  tests.   Yet  one  of  the  few  findings  that 
consistently  emerges  in  several  studies  is  that  nonverbal 
tests  enhance  differences. 

Howard  [34]  provides  an  excellent  conclusion  to  this 

section  on  assessment  center  advantages  with  the  following 

summary  of  possible  benefits: 


Help  with  the  Criterion  Problem- - Insta] lation  of  assess- 
ment procedures  may  force  better  job  analyses  and  identifi- 
cation of  the  important  criteria  for  success  on  a  job. 
Such  a  rigorous  process  has  been  aptly  described  in  con- 
nection with  an  analysis  of  the  job  of  foreman-  •  .  Another 
way  assessment  centers  may  help  with  the  criterion  problem 
is  by  training  assessors  to  evaluate  more  accurately  the 
performance,  behavior,  and  potential  of  others.   Assessors 
have  been  shown  to  have  greater  agreement  in  ratings  of 
different  assessee  traits  .  .  .,  but  it  has  not  yet  been 
demonstrated  that  assessors  will  experience  a  transfer  of 
training  in  rating  subordinates  under  the  unstandardized 
conditions  of  the  normal  work  experience. 

Training  Assessors  - -Benefits  of  assessor  training  have 
been  claimed  not  only  in  the  form  of  a  partial  solution  of 
the  criterion  problem  but  through  (a)  improvement  in  inter- 
viewing skills,  (b)  broadening  of  observation  skills, 
(c)  increased  appreciation  of  group  dynamics  and  leadership 
styles,  (d)  new  insights  into  behavior,  (e)  strengthening 
of  management  skills  through  working  with  simulations,  and 
(f)  broadening  one's  repertoire  of  responses  to  problems. 
No  well-designed  training  studies  have  validated  these 
promises,  however;  as  has  been  pointed  out  previously, 
firms  do  considerably  more  management  research  on  selection 
than  on  training  and  development  .... 

Development  of  Assessees- -Since  many  exercises,  like 
the  in-basket  and  oral  presentations,  were  used  formerly 
as  training  exercises,  many  assume  they  serve  such  purposes 
in  assessment  centers,  even  without  immediate  feedback  of 
results  t  m  _  Again,  evidence  supporting  this  training 

benefit  has  not  been  convincingly  provided.   Claims  of 
increased  self-insight  .  .  have  not  been  evaluated  with 
pretests  or  control  groups,  and  statements  by  assessees 
that  they  felt  the  program  was  informative  and  useful  for 
self -development  .  .  .  cannot  be  accepted  as  firm  empirical 
demonstrations  of  the  developmental  value  of  assessment 
centers . 

Minority  Group  Selection- -Recent  government  interest 
in  tEe  fairness  of  selection  tests  for  minorities  has 
stressed  that  selection  procedures  must  be  job  related, 
and  the  simulated  aspects  of  assessment  centers  do  have 
face  validity  in  this  respect.   One  study  bearing  on  this 
problem  at  AT$T  .  .  .  demonstrated  that  there  were  highly 
significant  correlations  between  performance  in  a  one  day 
approximation  of  the  company's  longer  Personnel  Assessment 
Program  and  performance  in  the  latter,  regardless  of  race 
or  sex.   The  research  design  in  this  study  more  closely 
resembled  alternate-form  test  reliability  than  predictive 
validity,  however;  thus  the  minority  group  fairness 
question  for  assessment  centers  is  not  yet  sufficiently 
answered . 
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Face  Validity- -The  simulation  exercises  in  particular 
have  high  face  validity,  and  the  whole  process  has  been 
claimed  to  be  received  favorably  by  managers,  especially 
those  who  may  be  mistrustful  of  tests  ....  Some  assessee 
questionnaires  have  also  indicated  that  the  majority  consider 
the  procedures  useful  and  objective  .... 

Attitude  Changes--It  has  sometimes  been  claimed  that 
assessees  may  change  their  attitudes  in  the  direction  of  a 
clearer  understanding  of  some  of  the  problems  facing  the 
managers  and  the  necessity  for  making  some  unpopular 
decisions  .  .  ,  but  so  far  the  evidence  is  anecdotal. 

In  summary,  the  bonus  benefits  ...  of  assessment 
centers  sound  promising  but  are  largely  without  research 
support . 

4 .   Disadvantages  (Criticisms  of  the  concept) 

Hinrichs  [32]  reports  a  study,  involving  47  Ss,  where 
predictions  of  management  potential  obtained  from  a  two  day 
assessment  program  were  compared  with  predictions  obtained  by 
a  traditional  technique.   The  traditional  technique  consisted 
of  decisions  made  by  two  experienced  managers  who  were  provided 
with  the  Ss  personnel  records.   Although  the  managers  were 
encouraged  to  contact  the  Ss  supervisors  by  telephone  for  addi- 
tional information,  only  a  few  such  calls  were  made.   Results 
of  the  study  showed  that  the  managers'  predictions:   "... 
were  as  highly  correlated  with  the  assessment  center  data  as 
were  overall  ratings  from  the  two-day  program,  except  for 
ratings  dealing  with  interpersonal  behavior."   The  point  of 
Hinrichs'  study- -and  a  criticism  of  it  were  made  by  Howard  [34] 

Hinrichs  argued  from  this  study  that  the  expensive  two- 
day  assessment  program  may  be  unnecessary  when  much  of  the 
same  information  can  be  obtained  so  much  easier.   Inter- 
personal relationships  seemed  to  be  a  relatively  untapped 
area  in  the  traditional  system,  however.   Dunnette   .  .  . 
[82]  .  .  .  also  disputed  Hinrichs'  conclusions  in  that  his 
correlation  of  .46  still  left  nearly  80  percent  of  the 
assessment  ratings'  variance  unaccounted  for. 
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To  this  writer,  the  real  test  would  seem  to  be  how 
well  personnel  records  or  performance  appraisals  vs. 
assessment  ratings  can  predict  future  management  perfor- 
mance, not  how  well  personnel  records  can  predict  assess- 
ment ratings. 

Another  critical  article  is  that  by  Wilson  and  Tatge 

[70],   Their  main  thesis  parallels  that  of  Hinrichs,  e.g., 

that  other  instruments  (paper  and  pencil  tests,  clinical 

and  patterned  interviews,  and  interpretation  of  performance 

records)  can  riva]  assessment  center  results  at  a  much  lower 

cost,  stating  that: 

In  the  writers'  opinion,  there  is  simply  no  available 
or  published  evidence  to  support  claims  of  the  significant 
superiority  of  the  centers  over  traditional  methods. 

Although  the  authors  challenge  many  claims  made  for 
assessment  centers,  they  fail  to  offer  any  evidence  to  support 
their  challenges.   Like  Hinrichs,  they  conclude  that  the 
greatest  benefit  of  assessment  centers  is  for  early  identi- 
fication when  there  is  no  performance  record  to  provide  an 
alternative  evaluation. 

Odiorne  [98]  comments  on  a  study  of  the  AT§T  program 

done  on  a  sample  of  500  managers  some  of  whom  had  not  been 

assessed  and  some  of  whom  had  received  various  ratings  at  the 

assesssment  center. 

This  doesn't  prove  validity  since,  as  Bray  and  Campbell 
report,  the  assessment  center  is  rarely  overruled,  and  that 
is  what  it  proves  ...  A  high  assessment  center  rating 
seems  to  be  itself  a  criterion  for  success,  not  a  predictor 
of  success . 

This  was  a  valid  criticism  of  the  program  in  an  opera- 
tional environment,  but  it  completely  ignores  other  studies 
which  controlled  for  such  contamination.   Bray  and  Grant  [10] 

describe  one: 
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The  Bell  System's  Management  Progress  Study  (Bray,  1964) 
offers  a  unique  opportunity  to  study  the  assessment  process 
.  .  .  The  uniqueness  of  the  study  from  the  standpoint  of 
studying  the  assessment  process  arises  from  several  aspects 
of  the  study  design: 

1.   There  is  no  contamination  by  the  assessment  results 
of  the  subsequent  criterion  data.   Along  with  all  other  infor- 
mation collected  on  the  422  subjects  of  the  study,  the 
assessment  data  are  being  held  in  strict  confidence.   Thus 
the  judgements  of  the  assessment  staff  have  had  no  influence 
on  the  careers  of  men  being  studied. 

The  other  assessment  problem  cited  by  Odiorne  was 
the  effect  of  a  rating  of  failure  on  an  employee's  later  perfor- 
mance.  This  is  spoken  to  by  Kraut  and  Scott  [41].   They  found 
that  the  lowest  rated  group  of  candidates  had  a  mobility  rate 
one-half  to  two-thirds  that  of  the  total  and  therefore  concluded 
that  a  poor  assessment  will  slow  promotion  but  not  necessarily 
deny  it. 

As  with  the  previous  section,  Howard  [34]  provides  a 

succinct  and  comprehensive  summary  of  the  field  as  follows: 

The  Crown  Prince  or  Princess.   Those  who  do  outstandingly 
well  in  assessment  centers  may  find  that  they  have  become  a 
crown  prince  or  princess.   Management  may  treat  them  so  well 
that  their  future  success  becomes  a  self-fulfilling  prophecy, 
the  morale  of  those  without  royal  status  may  decline,  and 
the  validity  coefficients  for  the  assessment  center  process 
may  become  inflated.   No  research  has  substantiated  these 
potential  coronation  effects,  however. 

The  "Kiss  of  Death".   A  candidate  who  does  poorly  at 
an  assessment  center  may  feel  that  he  has  been  given  the 
kiss  of  death  as  far  as  his  future  with  the  company  is  con- 
cerned.  This  could  result  in  some  undesirable  attrition, 
since  the  candidate  may  be  quite  competent  in  the  job  he  is 
now  performing.   Research  on  turnover  of  assessees  so  far 
has  been  inconclusive. 

Stress.   If  a  candidate  gets  the  impression  that  his 
entire  career  is  on  the  line  based  on  a  few  days  "on  stage," 
the  stress  effects  could  be  quite  strong.   It  would  seem 
important  that  the  data  from  the  procedure  not  be  made  of 
the  pass-fail  variety  or  kept  too  long  in  an  employee's 
file.   On  the  other  hand,  defenders  of  the  procedures  reply 
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that  since  stress  is  a  typical  part  of  a  manager's  job, 
a  candidate  should  be  stressed  to  see  how  he  copes  with 
it.   It  would  still  seem  important  to  keep  stress  in  the 
exercises  with  limits. 

The  Nonnominee.   The  feeling  that  an  individual  may- 
be part  of  the  "out  group"  if  he  or  she  has  not  been  selected 
to  participate  in  the  assessment  process  (whicli  may  become 
a  status  symbol)  is  another  dimension  of  employee  attitudes 
that  needs  to  be  empirically  tested. 

The  "Organization  Man."  Some  have  raised  the  issue  of 
whether  or  not  assessment  centers  may  not  proliferate  the 
model  of  the  conforming  organization  man  and  serve  to 
eliminate  the  unusual  or  imaginative  managers  that  are 
believed  to  be  needed  in  the  future.   A  study  at  Sohio 
showed  that  assessments  correlated  negatively  with  con- 
formity for  one  small  sample,  however  ....  An  IBM  study 
indicated  that  supervisors  may  nominate  those  higher  on 
conformity  and  lower  on  independence,  but  that  the  assess- 
ment procedure  itself  does  not  select  this  type  of  individual 
.  .  ..  The  organization  man  may  be  the  other  side  of  the 
nonnominee  problem;  the  most  able  and  not  the  least  able 
may  be  denied  access  to  the  assessment  center.   The  impli- 
cation is  that  it  is  the  nomination  procedure  and  not  the 
assessment  procedure  that  creates  the  organization  man 
syndrome.   The  supervisory  nominations  should  perhaps  be 
supplemented  by  self -nominations ,  peer  nominations,  personnel 
records,  or  assessment  of  everyone  at  a  job  level  if  numbers 
are  not  too  large. 

Costs .   Estimates  of  costs  have  ranged  from  the  price 
of  a  few  meals  to  $5,000  per  candidate,  exclusive  of  staff 
salary  ....  Installation  costs  are  the  highest,  but  to 
these  must  be  added  assessors',  assessees',  and  psychol- 
ogists' time,  travel,  accommodations ,  and  meals,  plus 
materials,  from  rating  sheets  to  videotapes.   Various  cost 
saving  devices  might  include  completing  all  possible  pro- 
cedures before  arrival  at  the  center,  conducting  exercises 
on  company  property  over  weekends,  and  combining  small 
companies  with  similar  jobs  in  a  multiple  company  center, 
perhaps  in  a  synthetic  validity  paradigm.   In  the  end, 
these  costs  must  be  weighed  in  the  context  of  current 
selection  ratios  against  the  possible  gains  in  selection 
and  training  in  some  kind  of  a  utility  model. 

In  summary,  the  possible  negative  outcomes  of  assess- 
ment centers,  have  much  the  same  status  as  the  (benefits), 
they  appear  reasonable,  but  for  the  most  part  they  lack 
supporting  data. 
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5.   Design  Considerations 

This  section  will  review  those  factors  seen  as  relevant 
to  the  design  of  an  assessment  center.   In  this  area,  Finkle 
and  Jones  [85]  ,  the  Foundation  for  Research  on  Human  Behavior 
[112]  and  Olmstead,  et.  al.  [122]  are  the  most  pertinent  and 
comprehensive  references. 

a.  Dimensions  to  be  Assessed 

The  three  sources  just  cited  cover  this  topic  quite 

thoroughly.   Miller  [50]  and  Kraut  [40]  offer  general  comments 

on  how  to  ascertain  relevant  job  dimensions.   Kraut  offers  some 

particularly  cogent  advice: 

Most  assessment  programs  are  not  based  on  an  empirical  study 
of  the  manager's  role,  as  might  be  done  through  careful  job 
analysis  or  a  critical  incident  study.   At  best,  they  are 
based  largely  on  a  review  of  the  research  literature  and 
the  judgment  of  executives  in  the  organization  as  to  what 
makes  for  an  effective  manager.   At  worst,  tiiey  tend  to 
be  copies  of  programs  in  other  companies. 

Helme  et.  al .  [114]  describe  the  development  of 

a  combat  simulation,  and  Krumm  et.  al .  [118]  writes  on  a 

research  project  to  explore  tactical  military  decision  making. 

Although  these  articles  are  quite  narrow  in  scope,  they  both 

deal  with  designs  of  military  simulations  and  should  be  of 

some  value  to  an  assessment  program  designer. 

b.  Number  and  Character  of  Assessments 

Several  sources  in  the  field  suggest  that  more 
than  one  assessment  of  an  individual  is  needed  during  a  career. 
The  basis  underlying  such  a  concept  is  outlined  in  Hall's  paper 
[30]  "A  Theoretical  Model  of  Career  Subidentity  Development 
in  Organizational  Settings."   Hall. defines  career  subidentity 
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.  .  the  person's  conception  of  himself  in  his  career 

role."   He  goes  on  to  explain  that  subidentity  (occupational) 

selection  "...  is  not  simply  a  matter  of  selecting  a  role 

but  also  one  of  choosing  an  aspect  of  one's  self  which  is 

potentially  effective  and  highly  valued."   He  underscores  the 

importance  of  this  decision  by  pointing  out  that: 

.  .  .  the  career  subidentity  grows  at  an  enormously  faster 
rate  in  the  first  year  than  in  subsequent  years  .  .  .  the 
early  years  undoubted] y  represent  a  critical  period  for 
learning  and  leave  a  lasting  imprint  on  the  person's  atti- 
tudes and  aspirations.   The  expectations  and  standards 
internalized  at  this  time  will  probably  be  enduring. 

These  comments  indicate  a  need  for  accurate  assess- 
ment at  the  point  of  occupational  selection  (career  entry) . 

Hall  then  introduces  the  concept  of  career  develop- 
ment which  he  defines  as  a  "  .  .  .  spiraling  combination  of 
career  choice,  subidentity  growth,  and  commitment."   He 
emphasizes  the  difference  between  occupational  choice  and 
career  choice,  explaining  it  as  follows: 

Occupational  choice,  the  choice  of  a  career  role,  is  made 
but  two  or  three  times  in  most  lifetimes,  while  career 
choice,  any  choice  which  will  affect  one's  career  develop- 
ment, can  occur  continually. 

He  points  out  that: 

.  .  .  this  focus  on  pos tselection  career  development  repre- 
sents a  departure  from  the  usual  psychological  concern  for 
occupational  choice--the  process  of  choosing  a  career  role 
congruent  with  characteristics  of  the  individual. 

These  latter  comments  indicate  that  the  need  for  assessment 

does  not  end  once  the  occupational  choice  is  made,  but  continues 

to  exist  as  a  result  of  the  career  choices  which  "can  occur 

continually. " 
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Pomerleau  [53]  expresses  the  same  basic  concepts 

in  a  more  pragmatic  way: 

When  we  speak  of  an  "identification  system,"  we  need  to 
be  reminded  that  it  has  two  subsystems:   (1)  Early  identi- 
fication of  high  potential  (EIHP)  and  (2)  Identification 
of  higher  potential  within  the  midmanager  ranks.   Funda- 
mentally, the  first  subsystem  asks:   What  could  the  raw 
candidate,  devoid  of  any  meaningful  performance  record 
beyond  academic  achievements,  do, given  the  opportunity  to 
perform  in  challenging  tasks?   The  second  subsystem  asks: 
What  has  the  midmanager  done  so  far  to  distinguish  himself? 
Our  source  of  data  can  be  culled  from  job  performance, 
experience,  developmental  assignments,  supervisor  appraisal, 
etc.,  and  then  determine  what  kinds  of  plans  could  be 
mapped  out  for  him  to  realize  the  full  measure  of  his 
managerial  potential. 

Slevin  [59]  has  a  slightly  different  view  of  the 
concept  and  is  the  first  to  specifically  describe  the  "assess- 
ment center  approach:" 

The  assessment  center  is  conducted  in  the  usual  fashion 
with  an  emphasis  on  both  evaluation  and  the  diagnosis  of 
developmental  needs.   These  two  variables  may  then  be  fed 
into  an  overall  manpower  plan.   This  plan  takes  into  account 
the  environmental  demands  on  the  firm  and  the  projections 
of  future  manpower  needs  in  various  areas.   Based  on  his 
own  competence  and  what  his  developmental  needs  are,  an 
individual  will  be  plugged  into  two  plans:   a  promotion 
program  that  is  specifically  constructed  for  the  individual, 
and  a  management  development  program.   The  promotion  pro- 
gram specifies  a  career  path  that  the  individual  is  likely 
to  follow  in  the  organization.   The  management  development 
program  focuses  on  the  difference  between  the  individual's 
current  competence  and  future  needs  as  defined  by  his 
tentative  career  path.   Using  both  the  promotion  and 
management  development  programs,  the  organization  can 
implement  an  effective  and  farsighted  manpower  plan. 

The  assessment  center  is  probably  the  best  example  of 
a  successful  "broad  band"  approach.   Because  it  is  able  to 
fill  both  organizational  appraisal  and  developmental  needs, 
the  assessment  center  is  likely  to  flourish  as  we  prepare 
our  organizations  for  the  1980' s. 

Miller  [50]  makes  one  of  the  most  explicit  state- 
ments on  the  subject: 
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Moreover  it  is  an  error  to  assume  that  an  individual's 
capacity  for  promotion  to  the  next  higher  level  or  to  jobs 
several  levels  higher  remains  constant.   Job  requirements 
change,  organizational  goals  and  demands  shift,  and  an 
individual's  personal  qualifications  are  likely  to  change 
with  time;  hence  his  potential  for  promotion  to  various 
jobs  will  vary  in  time.   Therefore  potential  must  be  period- 
ically reassessed. 

In  conclusion,  Hinrichs  [32]  summarizes  the  diverse 

objectives  of  an  assessment  program  as  follows: 

.  .  .  one  hopes  the  assessment  program  will  be  able  to 
identify  promotable . people  earlier  in  their  careers,  that  • 
it  will  help  to  clarify  some  of  the  skills  important  in 
promotion,  and  that  it  will  perhaps  identify  some  people 
who  should  be  promoted  but  who  might,  under  normal  circum- 
stances, be  overlooked.   Hopefully,  also,  the  program  will 
fulfill,  to  some  extent,  a  personal  development  function 
by  providing  practice  in  group  situations,  individualized 
feedback  regarding  observed  strengths  and  weaknesses,  and 
greater  understanding  regarding  the  caliber  of  the  competi- 
tion participants  are  up  against. 

c.   Uses  of  Results 

The  output  or  result  of  the  assessment  process  can 

be  utilized  in  several  different  ways.   This  is  not  to  imply 

that  a  single  program  could  meet  all  of  the  diverse  objectives, 

for  some  of  the  objectives  are  in  conflict,  e.g.,  selection 

usually  calls  for  stable  variables  to  be  measured,  while 

unstable  variables  are  the  focus  of  developmental  programs. 

Because  of  this,  it  is  important  that  objectives  (primary  and 

secondary)  be  specifically  delineated  before  the  program  is 

designed,  to  ensure  highest  efficiency. 

Tinkle  and  Jones  [85]  cover  this  topic  most 

thoroughly  in  Chapter  15  of  their  book.   They  identify  the 

following  uses: 

Selection  decisions 
Hiring 
Firing 
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Transfer  and  placement 

Organization  requirements 
Individual  requirements 

Management   development 

Organizational  planning 

Counseling  and  consulting 

Additional  uses  suggested  by  Culclasure  [108] ,  are 

career  motivational  prediction  and  selection.   Although  his 

study  failed  to  develop  predictors,  lie  wrote: 

Available  evidence,  however,  suggested  that  a  career  moti- 
vation screening  procedure  could  be  developed  .  .  .  Such 
a  procedure  would  employ  (a)  those  portions  of  the  Strong 
Vocational  Interest  Blank  (SVIB)  which  have  been  shown  to 
have  moderate  validity  (0.25)  for  predicting  Naval  officer 
retention  when  scored  with  the  Navy  Officer  Key,  and  (b) 
a  revised  version  of  the  Importance-Possibility  Scale. 

Helme,  et.  al.  [114]  also  address  military  assess- 
ment applications  as  follows: 

Early  identification  of  officer  leaders  and  development 
of  officer  leadership  from  cadet  training  through  company 
and  field  grade  assignments  are  of  major  concern  in  the 
management  of  the  Army's  manpower  resources.   The  Behavior 
and  Systems  Research  Laboratory  (BESRL)  conducts  research 
to  provide  scientific  means  of  identifying  individuals  with 
good  leadership  potential  for  officer  training,  selecting 
officers  for  commissioning,  and  evaluating  their  performance 
The  present  series  of  publications  records  the  conduct  of 
a  long-term  experimental  program  to  improve  the  basis  for 
selecting  and  developing  officer  leaders  in  accordance  with 
their  capability  to  meet  differing  leadership  requirements. 
Differential  prediction  and  evaluation  have  become  dominant 
objectives  in  the  effort  to  channel  officers  into  appropri- 
ate assignments  and  develop  their  potential  so  as  to  make 
best  use  of  their  abilities. 

As  mentioned  previously,  a  single  assessment 

program  is  realistically  confined  to  one  or  two  objectives. 

This  is  illustrated  by  the  fact  that  studies  of  operational 

assessment  centers  rarely  address  more  than  two  objectives. 

The  following  examples  are  offered. 
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Miller  [50]  cites  selection  and  identification; 
Pomerleau  [53]  stresses  identification  with  a  secondary  function 
of  development;  Slevin  [59]  says  essentially  the  same  thing, 
stating:   "No  two  problems  are  more  crucial  in  today's  changing 
organizations  than  effective  management  appraisal  and  develop- 
ment."  Kraut  and  Scott  [41]  describe  a  combination  of  assess- 
ment and  development: 

The  assessment  program  consists  of  two  and  one  half  days  of 
assessment  activity  patterned  after  the  AT§T  model,  plus  two 
days  of  purely  developmental  activity  .  .  .  The  development 
portion  includes  lectures,  seminars,  and  classroom  discussions, 
as  well  as  a  feedback  interview. 

Jelks  [37]  makes  an  important  point  regarding  the 

use  of  assessment  center  information: 

The  program  is  not  designed  as  a  final  judgment  factor  in 
the  selection  of  management  personnel,  nor  does  it  replace 
the  supervisory  appraisal  of  current  job  performance.   It 
simply  makes  one  more  piece  o±  information  available  that 
can  be  fitted  into  a  record  of  employee  performance. 

Kraut  [40]  makes  the  point  just  stated,  and  another, 
equally  important  one:   "The  data  should  ...  be  discarded  for 
any  decision  making  purpose  after  two  years  or  so."   Interviews 
disclosed  a  similar  policy  at  Pacific  Bell  Telephone  with  a 
figure  of  about  three  years  mentioned  as  a  maximum  "life"  of 
data  for  selection  purposes.   In  contrast,  Finkle  and  Jones  [85] 
cite  a  period  of  "five  to  ten  years,  or  longer." 

The  final  source  in  this  section  makes  a  prediction 

on  the  future  uses  of  assessment  centers.   Bray  and  Moses  write: 

Byham's  .  .  .  [11]  .  .  .  review  of  personnel  research  activ- 
ities indicates  a  growing  interest  in  developmental  and 
placement  strategies  rather  than  on  selection  per  se. 
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After  deciding  on  the  objective(s)  of  an  assess- 
ment program,  the  designer  next  faces  a  myriad  of  decisions. 
The  more  basic  of  these  will  be  arbitrarily  lumped  together 
and  considered  in  the  next  section, 
d.   Basic  Considerations 

Three  factors  considered  basic  to  assessment  program 
design  will  be  reviewed.   These  factors  are:   (1)  "Signs  vs. 
samples,"  (2)  Stable   vs.  unstable  variables;  and  (3)  Statistical 
vs.  clinical  combination  of  assessment  results. 

(1)   Signs  vs .  Samples .   Bray  and  Moses  [76]  write: 

Wernimont  and  Campbell  .  .  .  [67]  .  .  .  argue  that  tests  and 
other  predictors  should  be  used  as  samples  rather  than  signs 
of  behavior.  In  other  Avords,  the  measures  we  use  to  predict 
should  be  measures  of  behavior. 

Howard  [34]  seconds  this  view  by  noting  that: 

A  unique  contribution  of  assessment  centers  is  the  inclusion 

of  situational  tests  in  the  assessment  battery.   The  rationale 
behind  using  such  exercises  is  that  they  simulate  the  type 
of  work  to  which  the  candidate  will  be  exposed  and  allow  his 
performance  to  be  observed  under  somewhat  realistic  conditions, 
Contrary  to  the  aptitude  test  approach,  samples,  not  signs 
of  behavior  are  used  for  prediction. 

Wernimont  and  Campbell [67]  contend  that  samples  are  superior 

to  signs  in  the  following  ways:   (a)  less  bias  due  to  faking 

and  response  sets;  (b)  less  discrimination  in  testing  because 

samples  are  more  relevant  and  fair  than  signs;  (c)  less  invasion 

of  privacy  with  samples.  As  to  the  relative  effectiveness  of 

"signs  vs.  samples,"  Bray  and  Grant  [10]  stated  that: 

The  data  reported  make  it  apparent  that  the  situational 
techniques  (group  exercises  and  ln-Basket)  used  in  the 
Management  Progress  Study  produced,  despite  their  complex- 
ities, reasonably  reliable  results  and  that  they  markedly 
influenced  the  judgments  of  the  assessment  staffs.   The 
paper-and-pencil  instruments  (signs)  had  less  influence 
on  staff  evaluations  generally,  though  they  did  influence 
them  in  many  specific  ways. 
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Wollowick  and  McNamara  [71]  refer  to  the  preceding  source  and 

speculate  as  to  the  cost-effectiveness  of  simulations  (samples) 

An  important  question  frequently  raised  (Bray  anj  Grant, 
1966)  can  also  be  partially  answered  from  these  (Wollowick 
and  McNamara' s)  results.   That  is,  can  the  assessment  pro- 
cedure be  justified  in  light  of  its  additional  cost  and 
time  compared  to  the  use  of  paper-and-pencil  tests  alone? 
.  .  .  Inclusion  of  the  elements  unique  to  the  assessment 
center  procedure  .  .  .  nearly  doubles  the  criterion  variance 
accounted  for.   This  indicates  that  the  assessment  procedure 
makes  a  substantial  unique  contribution  to  the  prediction 
of  management  success. 

(2)  Stable  vs.  Unstable  Variables.   The  considera- 
tion given  to  this  question  in  the  design  of  the  Sohio  program 
is  described  by  Finkle  and  Jones  [85] : 

.  .  .  the  emphasis  in  the  program  at  Sohio  was  on  the 
development  of  information  that  might  be  expected  to  be 
reasonably  stable  and  consistent  over  some  period  of  time, 
perhaps  five  to  ten  years  or  longer.   Studies  have  been 
made  as  to  the  degree  to  which  measurements  and  judgments 
of  abilities,  interests,  attitudes,  and  values  of  individuals 
tend  to  change  over   time.   It  seems  appropriate,  in  cstablishi] 
a  group  of  basic  abilities  and  characteristics,  to  make  use 
of  available  studies,  general  experience,  and  common  sense 
to  eliminate  from  consideration  variables  that  are  highly 
vulnerable  to  change  during  a  relatively  short  period  of 
time. 

However,  this  approach  clearly  presupposes  that  the 
program  will  establish  less  information  pertinent  to  training 
and  developmental  plans  than  might  otherwise  be  possible. 
Immediately  the  question  arises:   Is  it  not  possible  to  do 
both,  namely  evaluate  some  characteristics  not  likely  to 
change  and  others  more  likely  to?   The  answer,  of  course, 
is  "yes."   Both  approaches  could  be  taken,  but  the  orienta- 
tions are  essentially  in  conflict.   Our  emphasis  in  Sohio 
was  placed  on  stability.   It  was  not  planned  to  reschedule 
the  elaborate  assessment  procedures  of  this  sort  in  any 
short  period  of  time  since  it  was  felt  that  it  was  important 
to  be  as  fully  aware  as  possible  of  more  permanent  charac- 
teristics when  making  manpower  decisions,  particularly  those 
of  a  long-range  nature. 

( 3 )  Statistical  vs.  Clinical  Combination  of  Assess  - 

ment  Results.   Bray  and  Grant  [10],  in  describing  the  AT§T 

Management  Progress  Study  assessment  program,  make  a  rather 

one-sided  statement: 
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Assessment  procedures  also  contrast  with  psychometric 
ones  in  the  Way  the  resulting  data  are  combined.   Psychometric 
approaches  depend  on  mathematical  methods  for  accomplishing 
this  purpose  whereas  assessment  approaches  combine  the  data 
judgmentally . 

Taft  [61]  is  more  objective  on  this  point  and  offers  an  excellent 

analysis  of  both  sides  of  the  question: 

We  have  argued  that  there  are  occasions  when  intuitive 
methods  of  making  predictions,  i.e.,  "clinical"  have  their 
appropriate  place.   Statistical  methods  cannot  be  used 
where  no  prediction  formula  exists.   But  some  personality 
assessors  speak  as  if  the  clinical  method  is  always  to  be 
preferred  as  it  enables  the  assessor  to  be  flexible  in  his 
use  of  the  data  in  a  way  that  is  not  possible  with  statis- 
tical techniques;  for  example,  the  clinician  can  give  weight 
to  obvious  but  rare  and  nonrepeatable  factors  in  the  subject's 
current  situation  which  could  not  be  validated  empirically. 
Other  advantages  claimed  for  the  clinical  against  the 
statistical  approach  are  that  it  docs  not  violate  the  essential 
unity  of  the  subject's  personality,  and  that  it  enables  the 
use  of  empathy  and  recipathy  in  making  the  predictions. 
(Actually  these  subjective  clues  could  also  be  used  as  data 
by  the  statistician  along  with  other  more  objective  data.) 

Other  assessors  regard  clinical  techniques  as  only  a 
last  resort.   A  number  of  advantages  can  be  quoted  for 
statistical  prediction  over  clinical,  most  of  which  boil 
down  to  the  fact  that  the  statistician  lias  a  far  more  effic- 
ient memory  and  a  larger  attention  span  than  the  clinician; 
he  can  "remember"  the  relevant  data  at  the  appropriate  time 
and  combine  them  with  other  data  in  order  to  obtain  optimal 
weightings  for  future  predications. 

And  so  we  have,  on  the  one  hand,  the  efficient  but  rigid 
and  inhuman  statistical  prediction,  and  on  the  other,  the 
flexible  and  humane  but  inefficient  clinical.   Which  one  is 
more  useful  in  personality  assessment? 

The  choice  of  method  will  depend  on  both  the  requirements 
and  the  over-all  situation,  including  sometimes,  public 
relations  considerations.   The  final  selection  of  assessment 
techniques  is  likely  to  be  a  mixture  of  both  subjective 
and  objective,  but  the  circumstances  that  will  favor  one  or 
the  other  at  any  stage  are  rather  vague,  and  the  choice  is 
usually  made  on  subjective  grounds,  although  it,  too,  could 
be  made  on  the  basis  of  objective,  empirical  investigation. 
In  general,  objective  methods  are  to  be  preferred  as  far  as 
possible  as  they  maximize  accuracy,  but  practical  consider- 
ations of  economy,  convenience,  and  the  limitations  of  the 
situation,  dictate  the  wholesale  use  of  subjective  methods 
in  personality  assessment.   These  subjective  methods  may 
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have  high  validity  under  favorable  circumstances,  and  where 
the  assessors  are  familiar  with  the  criterion  situation, 
clinical  judgments  may  actually  be  more  accurate  than  any 
objective  methods  are  ever  likely  to  be  in  predicting  to 
criteria. 

Wollowick  and  McNamara  [71]  report  an  interesting  finding  from 

their  study: 

The  subjectively  derived  combination  of  the  variables  (overall 
rating)  correlated  .37  with  the  criterion,  while  the  statis- 
tical combination  gave  a  multiple  of  .62.   This  suggests  that 
instead  of  deriving  an  (overall  rating)  by  subjective  mean(s), 
it  might  be  done  more  profitably  by  a  statistical  procedure. 
This  should  greatly  increase  the  predictiveness  of  the 
program. 

Meehl  [96]  makes  a  very  positive  statement  on 

the  issue  as  he  summarizes  a  number  of  studies  on  this  subject: 

In  spite  of  the  defects  and  ambiguities  present,  let  me 
emphasize  the  brute  fact  that  we  have  here,  depending  upon 
one's  standards  for  admission  as  relevant,  from  16  to  20 
studies  involving  a  comparison  of  clinical  and  actuarial 
methods,  in  all  but  one  of  which  the  predictions  made 
actuarially  were  either  approximately  equal  or  superior  _Lo 
those  made  by  a  clinician.   (Italics  are  Meehl ' s . )   Further 
investigation  is  in  order  to  eliminate  the  defects  mentioned, 
and  to  establish  the  classes  of  situations  in  which  each 
method  is  more  efficient. 

Lest  Meehl' s  position  be  misconstrued  by  quoting  him  out  of 

context,  his  "thesis  in  a  nutshell"  is  quoted: 

There  is  not  convincing  reason  to  assume  that  explicitly 
formalized  mathematical  rules  and  the  clinician's  creativity 
are  equally  suited  for  any  given  kind  of  task,  or  that  their 
comparative  effectiveness  is  the  same  for  different  tasks. 
Current  clinical  practice  should  be  much  more  critically 
examined  with  this  in  mind  than  it  has  been. 

One  final  viewpoint  on  this  subject  is  offered. 

Howard  [34]  cites  yet  another  study,  and  makes  some  suggestions 

for  future  assessment  programs: 

Again  we  find  a  demonstration  of  the  usefulness  of 
clinical  measurement  but  the  superiority  of  mechanical 
combination  of  data  .  .  .  (Sawyer  [56]).  Although  one 
study  cannot  be  generalized  to  all  assessment  centers, 
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•  it  certainly  appears  advisable  for  other  centers  to  research 
the  hypothesis  that  mechanical  combination  of  data  may  improve 
predictions  even  more.   Should  this  prove  true,  once  research 
costs  were  recovered,  the  unit  cost  savings  of  reducing 
assessors'  time  could  be  substantial. 

e.   Instruments  and  Techniques 

The  designer  of  an  assessment  program  faces  a 

formidable  task  in  the  selection  of  instruments  and  techniques 

which  will  best  serve  the  objectives  of  the  program.   Compounding 

his  task  is  the  plethora  of  commercial  instruments  vying  for  his 

approval . 

Albright  e t .  a 1 .  [73]  offer  a  relatively  brief  but 

comprehensive  coverage  of  "the  use  of  psychological  tests  in 

industry."   Bray  and  Grant  [10]  provide  an  overview  more  relevant 

to  assessment  programs  with  their  listing  of  the  instruments 

included  in  the  Management  Progress  Study: 

The  methods  used  for  collecting  information  on  the 
personal  characteristics  of  the  participants  are  representa- 
tive of  those  used  generally  in  assessment  activities.   A 
listing  of  the  techniques  with  a  brief  description  of  each 
follows : 

Interview.   A  two -hour  interview  with  each  man  directed 
at  obtaining  insights  into  his  personal  development  up  to 
that  time,  work  objectives,  attitudes  toward  the  Bell  System, 
social  values,  scope  of  interests,  interpersonal  relation- 
ships, idiosyncrasies,  etc.. 

In-Basket .   A  set  of  materials  which  a  telephone  company 
manager  might  expect  to  find  in  his  in-basket.   The  items, 
25  altogether,  range  from  telephone  messages  to  detailed 
reports.   In  addition,  examinee  was  furnished  with  such 
necessary  materials  as  a  copy  of  the  union  contract,  organi- 
zation chart,  and  stationery.   He  was  given  three  hours  in 
which  to  review  the  materials  and  take  appropriate  action 
on  each  item  (by  writing  letters,  memos,  and  notes  to  himself). 
Following  completion  of  the  "basket"  he  was  interviewed  con- 
cerning his  approach  to  the  task,  his  reasons  for  taking 
the  actions  indicated,  and  his  views  of  his  superiors, 
peers,  and  subordinates  (as  inferred  from  the  materials). 
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Manufacturing  Problem  (inade  available  by  John  Hemphill 
of  the  Educational  Testing  Service) .   A  small -bus iness 
game  wherein  the  participants  assumed  the  roles  of  partners 
in  an  enterprise  manufacturing  toys  for  the  Christmas  trade. 
The  participants  were  required  to  buy  parts  and  sell  finished 
products  under  varying  market  conditions,  to  maintain  inven- 
tories, and  to  manufacture  the  toys. 

Group  Discussion.   Also  a  leaderless  group  situation, 
focused  around  a  management  personnel  function.   Participants 
were  instructed  to  assume  the  roles  of  managers,  each  having 
a  foreman  reporting  to  him  considered  capable  of  promotion. 
Participants  were  required  to  discuss  the  merits  and 
liabilities  of  their  hypothetical  foremen  and  to  reach  a  group 
decision  regarding  their  relative  promotabilities . 

Proj  ectives .   (a)  Rotter  Incomplete  Sentences  Blank 
(published  by  the  Psychological  Corporation) .   (b)  Bell 
Incomplete  Sentences  Test  (by  Walter  Katkovsky  and  Vaughn 
Crandall  with  the  advice  and  assistance  of  Julian  Rotter)  . 
(c)  Thematic  Apperception  Test  (published  by  Harvard 
University  Press).   Six  of  the  cards  from  this  test  were 
administered. 

Paper-and-pencil  Tests  and  Questionnaires.   (a)  School 
and  College  Ability  test,  form  1  (published  by  the  Coopera- 
tive Test  Division  of  the  Educational  Testing  Service). 
Annually  since  1956  the  Personnel  Research  Section  of  ATqT 
has  developed,  following  the  Educational  Testing  Service 
format,  its  own  version  of  this  test.   (d)   Edwards  Personal 
Preference  Schedule  (published  by  the  Psychological  Corpor- 
ation) .   (e)  The  Guilford-Martin  Inventory  of  Factors  GAMIN 
(published  by  the  Sheridan  Supply  Company) .   (f)  Opinion 
Questionnaire,  Form  B.   Unpublished,  this  questionnaire, 
made  available  to  the  Bell  System  by  Irving  Sarnoff  of  New- 
York  University,  is  designed  to  reflect  a  person's  attitudes 
toward  making  money  and  advancing  himself. 

Miscellaneous .   (a)  Personal  history  questionnaire, 
(b)  Short  autobiographical  essay.   (c)  Q  sort  (70  items, 
self-descriptive) . 

Another  overview  can  be  found  in  Finkle  and  Jones 

[85].   They  provide  the  following  informative  table: 
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Figure  7 
Suggested  Factors  Indicating  the  Value  of  a  Test  or  Procedure 
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Source:   Finkle  and  Jones  [85] 
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They  continue  with  some  additional  suggestions  to  consider  in 
the  selection  or  design  of  tests: 
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In  projective  tests  the  psychologist  is  considered  to 
be  part  of  the  test.   Therefore,  the  choice  and  make-up 
of  projective  tests  should,  to  a  considerable  extent,  be 
left  in  the  hands  of  the  clinical  psychologist  who  uses 
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them.   However,  it  may  be  well,  in  choosing  the  clinical 
psychologist  who  will  particpate  in  the  program,  to  consider 
whether  the  projective  tests  he  prefers  to  use  will  prove 
generally  acceptable  to  assessees  and  managers  in  the  organi- 
zation and  will  be  acceptable  to  other  clinical  psychologists 
who  may  help  him  or  replace  him  in  the  program.   Ke  doubt, 
for  example,  that  ink-blot  or  word-association  tests  are  as 
acceptable  in  business  as  thematic  or  sentence-completion 
tests.   And  if  the  sentence-completion  procedure  is  to  be 
used  (particularly  if  one  is  to  be  tailor-made  for  the 
program) ,  it  becomes  important  to  have  some  assurance  that 
it  is  acceptable  to  psychologists  other  than  the  one  who 
designed  it. 

Multiple-Choice  Personality  and  Interest  Tests 

This  form  of  testing  has  been  the  principal  focus  of 
negative  criticism  of  management  level  testing  in  business. 
Accusations  have  ranged  from  unfairness,  through  perpetuation 
of  conformity,  to  simple  ridiculousness.   Anyone  considering 
the  use  of  such  procedures  should  obtain  good  professional 
counsel  since  much  of  the  criticism,  though  so  frequently 
and  sensationally  expressed  as  to  alienate  most  professionals, 
has  some  basis  in  fact. 

Two  characteristics  of  such  tests  probably  account  for 
most  of  the  criticism  leveled  at  them.   Tirst,  they  rather 
obviously  can  be  faked.   This  by  no  means  necessarily 
results  in  a  distortion  that  is  favorable  to  the  "faker." 
It  does,  however,  reduce  his  confidence  in  the  interpretation 
of  the  test  and  therefore  in  the  fairness  and  value  of  the 
assessment  process.   Furthermore,  it  may  prevent  the  effec- 
tive use  of  the  scoring  and  weighting  system  originally 
developed  for  the  test  without,  at  least,  a  follow-up  study 
to  examine  the  effects  of  attempted  faking  on  the  scoring 
system  that  was,  in  most  instances,  developed  from  answers 
given  by  persons  taking  the  test  experimentally  or  for 
counseling  purposes.   Second,  the  basis  of  development  and 
scoring  of  such  tests  is  actuarial  rather  than  clinical. 
This  means  that  the  choice  of  items  and  the  scoring  system 
are  ultimately  based  on  statistical  relationships  between 
actual  answers  to  the  questions  and  one  or  more  other 
measures  such  as  judgments  of  performance  or  personality 
constructs.   This  relationship  is  accepted  as  an  empirical 
fact  and  efforts  at  making  a  rational  interpretation  of 
individual  item  relationships  are  usually  not  attempted 
or  considered  relevant.   Consequently,  anyone  judging  the 
contribution  of  the  test  by  reading  and  thinking  through 
the  answers,  or  by  examining  the  weights  given  the  answers, 
may  simply  miss  the  mark  in  evaluating  the  test. 
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These  characteristics  of  multiple-choice  personality- 
tests  point  to  the  importance  of  a  careful  examination  of 
their  value  under  conditions  of  actual  use  as  a  means  of 
weighting  their  contribution  against  their  dubious  accept- 
ability.  In  selecting  tests  a  review  of  the  questions  for 
study  can  give  some  clue  as  to  the  reaction  that  can  be 
anticipated  by  test-takers. 

Interviewing 

Interviewing  can  presumably  be  called  a  type  of  assess- 
ment procedure,  but  there  seems  to  be  nearly  as  many  ways 
of  interviewing  as  there  are  interviewers.   As  discussed 
earlier,  arguments  can  be  made  for  having  all  interviews 
in  an  assessment  program  conducted  by  psychologists  or 
other  specially  trained  persons.   Other  arguments  support 
the  value  of  having  managers  conduct  the  interviews  within 
the  frameworks  of  some  assessment  programs.   The  Sohio 
program  adopts  the  latter  approach  and  in  fact  calls  for 
a  constantly  changing  (from  program  to  program)  group  of 
managers  to  conduct  interviews  with  a  minimum  of  special 
instruction  or  training. 

Clearly,  the  value  of  interviewing  under  such  conditions 
comes  from  maximizing  the  already  developed  habits  of  the 
managers  rather  than  in  attempting  to  change  or  "perfect" 
their  skills  in  accordance  with  some  predetermined  style  or 
pattern  of  interviewing.   In  a  brief  "orientation"  meeting 
prior  to  their  conducting  interviews,  the  managers  are 
asked  to  review  personal  history  forms  of  the  four  individuals 
each  of  them  will  interview.   It  is  pointed  out  that  these 
interviews  differ  from  ones  they  have  held  in  the  past  in 
that  this  time  "they  have  no  job  to  describe  or  "sell."   They 
are  encouraged  to  look  for  answers  in  the  personal  history 
form  that  can  be  used  to  encourage  the  interviewee  to 
elaborate  on  why  he  did  a  certain  thing  such  as  choose  a 
college,  a  major  in  college,  a  company  to  work  for,  or  a 
change  of  company.   They  are  also  asked  to  point  out  to 
the  interviewee  their  unique  role  as  his  representative  to 
the  assessment  committee:   How  would  the  interviewee  like 
to  be  represented?   What  does  he  feel  he  is  best  suited 
to  do  in  the  company  and  why  does  he  feel  he  is  so  suited? 
The  managers  are  cautioned  that  some  interviewees  may 
attempt  to  turn  questions  back  on    them  and  it  is  suggested 
that  they  refrain  from  describing  their  own  backgrounds 
and  activities  until  the  formal  interview  it  over. 

A  few  sample  interview  reports  from  past  programs  (edited) 
are  presented  and  it  is  stressed  that  their  own  reports  should 
contain  two  things:   information  about  the  interviewee  and 
their  judgment  about  him  from  the  interview  exposure.   No 
pattern  is  recommended  in  organizing  the  report.   In  fact, 
one  sample  interview  report  is  written  chronologically  from 
childhood  while  the  other  begins  with  the  first  Sohio  job 
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and  uses  earlier  experience  information  to  support  and 
elaborate  on  opinions  and  judgments  about  professed  skills 
and  interests.   The  managers  are  offered  tape  recorders  if 
they  do  not  wish  to  write  out  their  interview  reports  in 
longhand.   This  approach,  though  it  is  only  slightly  struc- 
tured and  offers  little  or  no  advice  on  style,  has  resulted 
in  most  instances  in  the  production  of  effective  1-1/2  to 
2-1/2  page  typewritten  reports. 

Situational  Exercises 

So  far  as  we  know,  situational  exercises  have  to  be 
developed  or  borrowed  rather  than  ordered  as  published 
versions  through  a  catalogue.   Such  procedures  can  provide 
either  individual  or  group  activity.   The  overriding  objec- 
tive of  the  approach  is  to  produce  activity  based  on  business- 
like matters  which  can  be  reliably  judged  by  observers  or 
reviewers.   It  is  presumed,  though  sometimes  given  secondary 
attention,  that  the  judgments  (or  observations)  produced 
can  be  weighed  compatibly  with  other  information  to  produce 
meaningful  combined  judgments  helpful  in  future  selection, 
placement,  or  developmental  decisions.   A  critical  aspect 
of  any  situational  exercise,  therefore,  is  the  realism  (in 
regard  to  business  problems)  in  the  eyes  of  those  partici- 
pating in  it  and  in  the  eyes  of  those  judging  it.   Accordingly 
with  either  individual  or  group  exercises,  the  task  provided 
should  bring  the  assessee  as  close  as  possible  to  realistic 
problem- solving  and  should  require  as  little  i;role-piaying': 
as  possible.  Observers  should  be  able  to  hear  or  observe 
enough   to  give  meaningful  interpretation  to  what  occurs 
rather  than  to  have  to  infer  what  a  person  is  thinking  or 
feeling.   Some  thought  should  probably  be  given  to  estab- 
lishing a  variety  of  tasks  so  that  persons  with  varied 
backgrounds,  skills,  and  characteristics  find  opportunity 
to  demonstrate  their  talents  and  styles.   For  example,, 
while  one  group  exercise  might  call  for  quick  and  con- 
vincing argumentation  based  on  little  specific  content, 
another  might  call  for  some  quantitative  analysis  as  a 
prerequisite  to  discussion,  while  still  another  might, 
feature  sensitivity  to  the  feelings  of  others  as  a  basis 
for  appropriate  contribution.   The  exercises  might  also 
vary  in  the  amount  of  preparation  allowed  in  order  to 
contrast  planned  and  rehearsed  behavior  with  impromptu 
behavior . 

In  all  exercises,  the  determination  and  explanation  of 
exactly  what  the  observer  is  to  note  and  report  is  quite 
important.   In  individual  exercises,  the  observer  may  examine 
the  results  of  the  exercise  (as  in  the  in-basket  exercise); 
he  may  observe  individual  performance  (as  in  the  presentation 
of  formal  talks);  or  he  may  actually  question  the  assessee 
as  to  why  he  took  certain  actions  (again,  as  in  the  in-basket 
exercise).   In  some  individual  exercises,  observations  may 
not  be  made,  but- rather  scores  (or  counts)  may  be  obtaine 
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by  formula  and  reported  to  the  assessment  committee  (as 
in  a  stock  market  exercise  developed  and  experimented 
with  at  Sohio) . 

For  group  exercises,  observers  may  have  specific 
assignments  to  concentrate  on  the  performance  of  one  or  two 
of  a  group  of  individuals  who  are  engaged  in  a  leaderless 
group  discussion  (such  as  reviewing  a  case  report)  or  in 
a  simulated  work  activity  (such  as  a  marketing-manufacturing 
problem) .   Here  the  observer  may  be  given  considerable  or 
little  structure  ranging  from  using  a  checklist  to  mark 
what  he  sees  to  simply  writing  notes  about  what  he  sees. 
Such  notes  can  then  provide  him  with  the  basis  for  preparing 
a  written  report  or  for  "telling"  the  assessment  committee 
what  "his" assessees  did  during  the  exercise. 

An  observer  could,  of  course,  be  asked  simply  to  com- 
plete a  series  of  rating  scales  similar  to  or  even  identical 
to  the  skill  and  style  variables  to  be  used  by  the  assessment 
committee.   However,  we  feel  that  this  approach  has  two 
disadvantages.   First,  the  other  members  of  the  assessment 
committee,  who  concentrated  on  other  assessees  or  who  did 
not  observe  the  exercise  at  all,  will  have  no  Kay  of  joining 
in  the  interpretation  and  judgment  of  the  performance  of 
the  individual  being  reported  on;  they  must  simply  accept, 
or  not  accept,  -the  judgments  of  the  reporting  observer. 
Second,  the  observer  may,  by  rating  the  assessee  on  the 
variables  solely  from  observations  in  one  exercise,  over- 
commit  himself  on  each  rating  prior  to  hearing  evidence 
from  the  other  exercises  and  procedures.   To  offset  these 
possible  problems,  we  suggest  that  observers  be  given  a 
checklist  or  structure  that  produces  broad  coverage  and 
descriptions  without  calling  for  judgments  on  the  basic 
program  variables.   A  highly  detailed  checklist  seems 
advisable  only  when  the  observers  can  be  well  trained  in 
its  use- -generally  when  management  observers  are  expected 
to  remain  as  staff  contributors  through  several  programs. 

Though  somewhat  less  important  than  other  consideration.?, 
some  method  of  ready  identification  by  observers  of  all 
participants  at  a  group  exercise  fosters  better  association 
of  remarks  with  those  making  them.   We  have  adopted  the  use 
of  colored  vests --other  approaches  could  probably  serve  as 
well.   Observers  otherwise  will  have  no  trouble  identifying 
the  persons  they  are  assigned  to  concentrate  on- -but  they 
might  have  difficulty  in  identifying  exactly  "who  their 
assigned  assessees  are  talking  to  or  with  in  the  group- - 
particularly  if  a  couple  of  the  "other"  assessees  have  their 
backs  to  the  observer.   By  having  each  participant  in  the 
group  exercise  clearly  identified  (from  all  directions)  by 
color,  the  observers  can  merely  note  interactions  by  these 
symbols,  and  when  their  reports  are  later  typed,  the  typists 
can  substitute  the  correct  names  for  the  colors. 
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One  final  bit  of  advice  for  the  designer  is  offered 

by  Hardesty  and  Jones  [31],  e.g.: 

In  addition  to  the  variety  of  assessment  approaches  used 
in  the  program,  the  designers  purposely  attempted  to  build- 
in  duplication  when  possible  and  to  strive  for  some  overlap 
among  and  within  the  assessment  approaches  in  order  to 
increase  the  confidence  they  could  have  in  the  assessment 
information. 

The  remainder  of  this  section  will  continue  to 

consider  specific  instruments  and  techniques  individually. 

(1)  Projective  Personality  Tests.   The  best 
article  found  on  this  subject  is  "Contributions  of  projective 
techniques  to  assessment  of  management  potential,"  by  Grant  et . 
al .  [27] .   The  authors  conclude  that  "...  projective  reports 
particularly  influenced  the  assessment  staff  in  rating  such 
characteristics  as  work  motivation,  passivity,  and  dependency. 

(2)  Interviews .   A  number  of  pertinent  articles 
have  been  written  on  interviews.   Grant  and  Bray's  [26]  "Con- 
tributions of  the  interview  to  assessment  of  management,  potential," 
is  one  of  them.   They  found  that:   "Analyses  of   the  data 
clearly  indicate  that  information  from  the  interview  reports 
contributes  to  assessment  center  evaluations." 

"The  validity  of  personnel  interview"  by 
Ghiselli  [23]  is  a  brief  but  worthwhile  article.   His  conclusion 
states  that:  •  •  •  "the  ordinary  personnel  interview  is  not 
necessarily  and  invariably  invalid,  but  rather  that  its  validity 
may  be  at  least  equal  to,  if  not  greater  than,  the  validity 
of  tests." 

Prien  [54]  reports  a  study  to  determine  the 
effectiveness  of  interview  information  as  a  predictor  of  future 
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•  job  performance.   He  concluded  that: 

Psychologists  differentiate  individuals  through  interviews 
on  two  general  dimensions:   Personal  Relations  Effectiveness 
and  Intellectual  Functioning  .  .  . 

and 

Psychologists  can  predict  the  potential  job  performance  of 
individuals  on  the  basis  of  information  and  impressions 
obtained  in  an  interview.   The  interview  has  some  validity 
for  the  assessment  of  higher-level  personnel. 

(3)   Simulations/Situational  Tests.   This  category 

can  be  subdivided  into  group  and  individual  tests.   Those  articles 

dealing  with  the  entire  field  will  be  reviewed  first,  followed 

by  those  on  group  exercises.   Individual  tests  will  be  covered 

last. 

Finkle  and  Jones  [85]  again  offer  an  excellent 

treatment  which  is  enhanced  by  the  inclusion  of  advantages 

and  disadvantages  of  situational  techniques: 

This  approach  consists  of  one  or  more  standard  exercises 
expected  to  bring  out  ability  and  personality  characteristics 
directly  and  more  obviously  related  to  successful  performance 
in  supervisory,  administrative,  and  managerial  responsibil- 
ities.  These  exercises  may  consist  of  individual  tasks,  such 
as  writing  recommendations  about  a  variety  of  letters,  reports, 
notices,  and  so  on,  typical  of  the  contents  of  an  executive's 
"in"  basket.   Or  they  may  require  elaborate  group  activity 
of  six  or  eight  candidates  "running"  a  simulated  company 
or  "serving"  on  a  committee.   To  some  degree  how  well  the 
situations  are  handled  is  informative:   for  example,  the  best 
recommendations  on  the  in-basket  items,  higher  profits  in 
running  the  company,  or  controlling  the  committee.   There  is, 
however,  some  question  as  to  whether  or  not  results  on  these 
situational  exercises  guarantee  results  in  a  real  life  situ- 
ation.  Much  more  important  is  the  picture  obtained  of  the 
abilities,  skills,  styles,  and  personalities  of  the  partici- 
pants.  Such  characteristics  are  noticed  as:   (1)  degree  of 
organization,  (2)  willingness  or  reluctance  to  make  decisions, 
(3)  willingness  to  take  action,  (4)  clarity  and  impact  in 
communications,  (5)  awareness  and  sensitivity  to  the  thoughts 
and  feelings  of  others  (6)  impact  on  group  decision-making 
and  (7)  approach  to  problem  definition  and  attack. 
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The  interpretation  of  these  situational  activities  may 
vary  considerably  from  a  rather  intuitive,  commonsense,  or 
clinical  judgment  formed  by  either  psychologists  or  manage- 
ment personnel  as  to  the  significance  of  certain  answers  or 
actions  to,  on  the  the  other  hand,  a  rather  empirically 
related  analysis  of  the  answers  and  actions  as  compared 
with  other  judgments  in  terms  of  ratings,  later  actions  or 
future  consequences.   In  other  words,  these  techniques  may 
be  approached  on  the  basis  of  the  standardized  selection 
techniques,  such  as  regular  paper-and-pencil  tests,  supposedly 
supported  by  validation  studies  against  other  measures 
obtained  after  employment;  or  they  may  be  judged,  more  as 
the  clinical  information  is  judged  by  the  clinical  psycholo- 
gist. 

Advantages 

a.  The  problems  and  situations  used  with  this  approach  are 
designed  to  bring  out  characteristics  and  abilities  that 
seem  directly  related  and  relevant  to  success  in  the  business 
world. 

b.  These  techniques  involve  some  important  areas,  such  as 
interpersonal  relations,  administrative  skills,  and  communi- 
cations ability  that  are  not  well  covered  by  the  other 
techniques . 

c.  These  techniques  are  readily  adaptable  to  administration 
and  use  by  business  personnel  as  well  as  by  psychologists. 

d.  The  descriptions  of  the  styles   and  abilities  of  indi- 
viduals as  assessed  by  these  techniques  are  more  readily 
understandable  to  management  personnel  making  promotion 
decisions  than  are  some  of  the  descriptions  written  by 
psychologists  following  the  clinical  approach  or  scores 
derived  from  the  statistical  approach. 

Disadvantages 

a.  This  approach  can  be  relatively  time-consuming,  depending 
upon  the  number  of  exercises  employed.   Most  exercises  take 
from  one  to  three  hours. 

b.  There  is  always  the  possibility  in  the  group  exercises 
that  one  individual  will  look  relatively  strong  or  weak  in 
some  respects  because  of  the  makeup  of  the  particular  group 
in  which  he  is  placed.   This  and  the  possibility  that 
observers  may  not  be  consistent  in  how  they  judge  and  report 
what  they  see  make  the  process  one  of  low  reliability  (con- 
sistency) . 

c.  There  is  no  certainty  that  performance  under  testing 
conditions  will  transfer  to  later  performance  under  real 
conditions,  particularly  if  results  on  the  "correctness" 
of  the  answers  are  considered  rather  than  style. 
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d.  The  cost  of  this  technique  varies  according  to  the  method 
that  has  been  used  for  the  interpretations  of  the  results. 

If  the  empirical  approach  has  been  taken  to  simply  validate 
some  measurable  results  or  actions  or  answers  by  comparing 
them  statistically  with  later  judgments  such  as  ratings, 
promotions,  survival,   and  so  on,  the  cost  of  administering 
and  directly  interpreting  is  relatively  low.   However,  if 
the  approach  taken  toward  interpretation  of  the  situational 
or  job-sample  tests  is  one  of  a  commonsense  review  by 
management  as  to  the  adequacy  and  appropriateness  of  the 
actions  or  performance,  or  if  it  is  an  interpretation  by 
psychologists  of  personality  characteristics,  in  each  of 
these  circumstances  it  may  be  quite  costly  because  of  the 
hourly  salary  of  the  key  individual  making  the  interpretation, 
that  is,  either  the  manager  or  the  psychologist. 

e.  No  matter  how  this  technique  is  to  be  judged,  it  should 
be  employed  only  after  a  number  of  experimental  or  pilot 
sessions  have  been  conducted  and  analyzed. 

Parry  [52]  offers  insights  into  the  reasons 

why  simulations  seem  to  be  more  effective  than  traditional  tests 

The  use  of  games  and  simulations  enables  the  learner  to 
develop  in  three  major  areas.  Here  are  the  objectives  that 
apply  to  virtually  all  games  and  simulations. 

The  participant  sharpens  his  skills  of  analysis,  making 
judgments  and  experiencing  the  immediate  consequences  of 
them.   For  example,  he  learns  to  separate  relevant  from 
irrelevant  variables,  establish  priorities,  identify  assump- 
tions, separate  fact  from  opinion,  set  realistic  goals, 
assign  value  (i.e.,  weight  the  facts  according  to  their 
relative  importance) ,  distinguish  between  might  do  and  must 
do,  and  so  on.   Let's  summarize  this  objective  with  the 
phrase  Analytical  Thinking. 

The  participant  gains  insight  into  his  own  behavior  as 
he  interacts  with  others;  and  he  develops  sensitivity  to 
the  perceptions,  needs,  goals,  management  styles--in  short, 
the  behavior- -of  others.   To  this  end,  a  game  might  bo 
regarded  as  a  projective  device.   The  participants  are 
caught  up  in  the  spirit  of  the  game  and  play  it  without 
stopping  to  ask,  "I  wonder  what  this  move  will  shoA\;  about 
my  personality?   How  does  my  performance  in  the  game  relate 
to  my  performance  on  the  job?"   Once  the  game  is  over  and 
these  questions  come  out  in  discussion,  the  participant 
begins  to  realize  that  the  behavior  he  projected  during  the 
game  is,  more  likely  than  not.,  an  accurate  sample  of  his 
behavior  on  the  job.   We  might  summarize  this  objective 
with, the  phrase  Insight  and  Sensitivity. 
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The  participant  practices  and  refines  his  skills  in 
dealing  effectively  with  others.   Broadly  speaking,  we 
can  divide  the  work  of  managers  and  supervisors  into  two 
broad  areas:   people-handling  skills  (e.g.,  communications, 
human  relations,  training  and  developing  others,  etc.),  and 
task-handling  skills  (e.g.,  work  simplification,  planning, 
scheduling,  controlling,  etc.).   Both  of  these  elements 
are  present  in  most  games.   A  successful  manager,  of  course, 
is  effective  on  both  dimensions.   In  contrast,  the  new 
manager  tends  to  be  either  task-oriented  or  people-oriented. 
Thus,  we  might  regard  the  games  as  a  series  of  exercises  in 
which  each  participant  can  successively  develop  and  refine 
his  mixture  of  people -handling  and  task-handling  skills. 
We  could  use  the  phrase  Skills  Practicum  to  summarize  this 
obj  cctive . 

These  three  objectives  were  brought  home  very  nicely 
by  a  participant  who  remarked,  "These  games  are  sneaky.   I 
play  them  just  like  I  play  poker  or  any  other  game  ...  I 
play  to  win,  without  giving  much  thought  to  what  'style' 
I'm  using  or  what  my  alternatives  are.   But  then,  when  we 
sit  down  to  discuss  the  games,  I  begin  to  see  some  of  the 
other   ways  that  I  might  have  played  them.   In  fact,  I  now 
find  myself  giving  a  good  deal  more  thought  to  my  poker 
game.   I  guess,  in  a  way,  that  all  of  life  is  a  game,  and 
that  how  you  play  it.  is  just  as  important  as  whether  or  not 
you  win.   We  can't  all  be  winners  ...  at  least  not  all 
the  time.   But  we  can  play  a  good  game  all  the  time." 

Greenwood  and  McNamara  [2S]  report  their  study 
on  "Interrater  reliability  in  situational  tests."   Not  only 
did  they  find  acceptably  high  interrater  reliabilities,  but, 
more  importantly, :   "...  that  adequate  reliability  can  be 
obtained  from  the  use  of  nonprofessional  evaluators  in  business- 
oriented  situational  tests." 

Proceeding  to  articles  concerned  with  group 
exercises,  Streufert,  et . al .  [60]  make  the  transition  with 
their  description  of  "A  Tactical  Game  for  the  Analysis  of  Complex 
Decision  Making  in  Indviduals  and  Groups."   Their  simulation 
was  designed  to  maximize  differences  in  information-processing 
characteristics.   They  ascribe  the  differences  found  primarily 
to  variations  in  a  personality  and  environment.   This  instrument 
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is  briefly  described,  along  with  techniques  for  analysis  of 

results,  and  information  on  how  to  purchase  the  associated 

computer  program  is  provided. 

Helme  et.  al.  [114]  report  on  "Dimensions  of 

Leadership  in  a  Simulated  Combat  Situation."   This  simulation 

probes  different  dimensions  than  the  proceeding  one.   Although 

it  is  primarily  a  report  of  research,  the  description  of  the 

simulation  would  be  of  value  to  a  military  assessment  program 

designer.   Some  idea  of  the  flavor  of  a  combat  simulation  can 

be  obtained  from  this  excerpt: 

After  four  hours'  sleep,  the  officer  was  awakened  at 
2:30  and  told  to  report  to  MAAG  headquarters .   The  host 
nation  had  been  invaded  with  nuclear  strikes.   His  task 
for  the  next  eight  hours  xvas  to  direct,  by  radio,  four 
jeep-mounted  teams  in  a  reconnaissance  of  war  damage  and 
radiation  levels,  continually  receiving  and  recording 
information,  transmitting  orders  to  meet  emergencies,  and 
making  a  report  of  results  of  the  survey  to  his  superior. 
Of  all  the  technical  and  administrative  tasks,  this  one  was 
carried  out  under  the  most  sustained  pressure  and  fatigue. 

Jaffee's  book  [90]  contains  concise  but  com- 
plete examples  of  both  individual  (in-basket),  and  group 
(leaderless  group  discussion  and  business  game)  simulations. 
The  examples  are  complete  with  scoring  check-lists  and  forms. 

One  of  the  sources  most  relevant  to  military 
applications  is  the  report  on  "Development  of  Leadership  Assess- 
ment Simulations,"  by  Olmstead  et.  al .  [122].   This  is  a  detailed 
description  of  the  design  of  the  simulations  currently  used 
in  the  USAIS  Assessment  Center  pilot  project.   This  work  was 
done  under  contract  by  the  Human  Resources  Research  Organi- 
zation (HumRRO)  Division  No.  4  which  is  located  at  Fort 
Benning.   The  combination  of  design  philosophy,  problems 
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encountered  and  their  solution,  rating,  analysis  of  results 
and  assessor  training,  make  this  reference  especially  valuable 
as  a  guide  for  a  program  designer. 

The  in-basket  exercise  is  the  most  widely  used 
individual  simulation  and  the  literature  is  replete  with 
articles  on  this  topic.   The  stature  of  the  exercise  is  illus- 
trated by  this  quotation  from  Byham  and  Pentecost  [12] :   "How- 
ever, one  exercise,  the  in-basket,  seems  to  have  emerged  as 
the  most  important  exercise  of  most  assessment  centers."   This 
view  represents  a  consensus  expressed  by  other  writers. 

Frederiksen,  et .  al .  [22]  provide  a  description 
of  how  the  exercise  came  into  being  and  its  early  development. 
Examples  of  in-basket  materials,  scoring  and  validation  studies 
are  reported. 

The  landmark  article  in  the  in-basket  field  is 
Lopez'  "Evaluating  executive  decision  making,"  [119].   This 
source  covers  the  entire  spectrum  from  design  considerations 
to  application  and  analysis.   It  also  covers  the  previously 
mentioned  study  along  with  others  done  by:   The  Bell  System, 
the  Bureau  of  Business,  Harvard  Business  School,  Port  Authority 
of  New  York,  Sears  Roebuck,  and  General  Electric.   In  scope  and 
detail,  it  is  unmatched. 

Meyer  [48]  details  the  design  and  validation 
of  the  General  Electric  in-basket.   Although  the  test  itself 
is  not  included,  the  article  would  be  an  excellent  resource 
for  the  program  designer. 
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Jaffee  offers  two  books  containing  in-basket 
information  [90] ,  [89] .   The  latter  is  devoted  exclusively  to 
the  in-basket.   It  is  a  workbook  containing  two  complete  in- 
basket  exercises  with  all  forms,  instructions,  etc.  to  enable 
administration  to  individuals.   This  is  not  to  imply  that  it 
is  intended  to  provide  the  program  designer  with  an  off-the- 
shelf  instrument,  for  the  point  is  made  repeatedly  that  to 
be  effective  the  in-basket  must  be  tailored  to  a  specific 
program.   Rather,  the  book  was  intended  as  a  training  device 
for  students. 

f.   Reliability 

The  summary  of  reliability  studies  done  by  Howard 
[34]  negates  the  requirement  for  considering  them  individually 
She  writes: 

In  many  assessment  center  exercises  and  in  the  final 
evaluations  each  participant  is  evaluated  by  more  than  one 
assessor.   Accordingly,  interrater  reliability  becomes  a 
matter  of  some  importance,  in  addition  to  the  reliability 
of  individual  measures.   A  summary  of  reliability  data  is 
shown  in  Table  1, 

It  should  be  noted  that  the  AT§T  studies  of  the  in- 
basket,  projective  tests  and  interviews  probably  had 
inflated  reliability  estimates ,  since  the  interrater  relia- 
bility was  determined  for  the  written  report  of  a  procedure 
not  the  procedure  itself.   Two  raters  in  high  agreement  on 
what  a  report  says  is  a  far  less  potent  finding  than  two 
raters  in  high  agreement  on  how  a  candidate  performs  in  a 
situational  exercise.   The  reliability  coefficients  do 
indicate,  however,  that  the  reports  presented  clear  evalu- 
ations from  which  consistent  ratings  could  be  made. 

In  summary,  based  on  the  data  available,  interrater 
reliabilities  for  assessment  evaluations  and  for  several 
assessment  components  seem  sufficiently  high  to  support 
their  further  use.   There  appear  to  be  no  advantages  of 
ratings  vs.  rankings  or  psychologists  vs.  managers  in  terms 
of  reliability. 


4-J 


X> 


Pi 

U 
(D 

•M 

rt 

rH 

S-, 
(1) 
•P 
PI 


10 

rH 
O 
I/) 
tO 
0 

to 

tO 


tO 

CD 

rH 
rO 

03 
■H 
rH 

> 


d 
e 

o 

c_j 


II 

II 

^ 

u 

IrH 

4-» 

n 

*v 

cd 

cd 

d 
o 
o 

tO 

LO 

o 

co 

rH 

■zf 

LO 

O 

CD 

cr> 

CT\ 

ONO 

•o 

■«t 

["•» 

« 

• 

• 

• 

t^ 

.  vo 

• 

t-« 

• 

1 

1 

cd 

cd 

icd 

1 

• 

• 

• 

LTj 

CT\ 

lO 

CO 

en 

to 

**  to 

LO 

•^ 

#» 

». 

LO 

t-~- 

o 

O    CN) 

r-» 

r~- 

CO 

CT) 

vO  CO 

CO 

** 

to      •> 

to 

■h 

to 

r^ 

. 

vO 

•  cr> 

• 

• 

• 

• 

«      • 

• 

to 

bO  to 

IMl 

to 

bo 

• 

• 

• 

bo  d  m  d 

bO  d 

•\ 

-. 

*\ 

n 

c\ 

~ 

•*         r 

•V 

d 

•H    £ 

■H 

d 

H 

*s 

to 

». 

to     •> 

to 

to 

to 

to 

to   to 

to 

■H  ^  -H 

y 

•H  M 

to 

'■: 

to 

bO  to 

bO 

bO 

bO 

bO 

bO  W> 

bo 

P 

d  p 

d 

+-> 

d 

bC 

r—t 

to  d  bo 

dcd 

dcd 

d 

d    ■ 

d  d 

d 

cd 

cd   cd 

cd 

cd 

cd 

d 

H 

d 

H     C 

•H  LO 

■h  ai 

■H 

•H 

•H  -H 

•H 

ptj  Pt 

•H  ^ 

•H 

^  -H 

+->  CO 

+->   CO 

P 

+J 

+->   P 

+J 

4-> 

d 

4-> 

d  p 

cd      • 

cd     • 

nJ 

rt 

cd   cd 

cd 

, — -v 

/ — N 

/■ — \ 

cd 

- 

cd 

cd  cd 

oi, 

P< 

PA 

p* 

Pti  Pi 

Ptj 

cd 

Xi 

u 

eSPi 

C<  P<  P^ 

10 

to 

, 

to 

to 

to 

•p 

p 

to 

+-> 

+-> 

+-> 

(/! 

to 

bO 

to 

to 

to 

•H 

•H 

m 

d 

■H 

•H 

•H 

bo 

bO 

o 

•H    lO 

bo 

bO 

bO 

O 

to 

o 

to 

to  to 

to 

P    Si 

O 

O 

O 

i— i 

Sh 

i— i 

U 

U   U 

5-i 

to 

cd  a 

rH 

rH 

i— l 

o 

0) 

o 

tD 

a>  a> 

CD 

y> 

d  > 

o 

o 

o 

x; 

bO 

rd 

bO 

bo  bo 

DO 

•H 

in    >h 

r^ 

X, 

rd 

u 

ai 

u 

cd 

cd  cd 

cd 

cd 

o  cd 

U 

o 

u 

X 

d 

X 

d 

d  d 

d 

PUP    tO 

X 

X 

» — 

to 

rf 

to 

rt 

cd   cd 

cd 

rH  ,Q 

to 

to 

to 

P, 

e 

p. 

a 

e  m 

6 

rH 

cd   O 

p. 

& 

p, 

CX] 

tO 

CO 

to 

LO    LO 

LO 

< 

CD 

in) 

CN1 

CN1 

to 

to 

rH 

<D 

to 

rH 

to 

d 

fl 

O    rH 

• 

U 

to 

X 

to 

+-> 

o 

o 

bo  cd 

P 

5-t 

0 

O 

o 

<L> 

•H 

•H 

CD  -H 

bo 

O 

i-l 

5-< 

rH 

E-X 

to 

to 

I— 1 

rH 

-t->  p 

6 

>N 

m 

rH 

Ph 

rl 

<u  to 

a 

d 

cd 

cd 

crt    !-■ 

J-l 

0} 

fX 

"\ 

rH    Id 

a> 

a> 

•H 

•H 

U    CD 

i— i 

H 

ri4 

cu  "d 

3 

. 

t3 

• 

,QJ3 

6 

6 

P 

4-> 

4-> 

t— i 

i— 1 

to 

E  cd 

o 

bO 

cd 

bO 

O     i 

•H 

•H 

d 

d 

CNJ    O 

cd 

•H 

cd 

cd  cj 

fH   4-4 

cu 

tH 

r-    d 

tJ 

T3 

cd 

cd 

rH   PM 

in 

,Q 

H 

bO_i 

bOS 

i-J 

s 

Pun 

+-> 

P 

O 

cd 

tO 

to 

o 

O 

< — w — \ 

> 

/ — ^ 

, — \ 

/ — \ 

, — s 

/ ^ 

, — » 

t— 1 

rH 

PL, 

PL, 

cd  ja 

o 

cd 

X> 

U 

cd 

X 

u 

o 


(D 

U 

d 

5h 

o 

3 

to 

O 

6 

00 

o 

2: 


o 


o 


d 


pf~I 

ffi 

nn 

K 

o 

o 

o 

o 

CO 

CO 

CO 

CO 

r-. 
CD 

iM 

cd 

PL, 


H 


r~- 

r^ 

d 

d 

d 

O    11 

O    II 

o 

10 

to 

to 

6  2 

&  ^ 

E 

O  v— ' 

O  v— ' 

o 

rd 

X 

rC 

H 

H 

H 

II 


d 
d 
o  ^ 

u 


r-1 

CD 

cd 

PL,  r~^ 
CO 
U3"  •^- 
l 
rH  CN1 
rH  rH 
CD 

d 
d 

O 

u 
u 
2 


z 


pq 


cd 
cd 

e 

cd 

2; 

u 
2 

UTJ  ^— n 
CO 

Td  c^o 
O  cnj 

o 

£  II 

d 

CD  2 

CD  <— ' 
rH 


H 

off 
H 


■p 

d^ 
cdL^ 

L^LO 

cjto 
cdv-^ 

in 


88 


+-> 

•H 


•H 

T-i 

CD 
Pi 

fn 

<L> 
P 
c3 
f-i 
Pi 
CD 
P 

a 


CO 

u 

o 
to 

0> 
10 

to 
< 


■H 
Pi 

> 


cd 
u 
Pi 

o 


0) 

bO 

<Drt 

rt 

f-H    0> 

<* 

fH    bO 

o> 

O    <1) 

• 

O  .-( 

i 

t-H 

LO 

(si    O 

OO 

oo    (J 

• 

•    i 

C 

■h 

II     o 

tO 

a 

bO 

c 

C 

oS  cxi 

•H 

•H  r^ 

P 

tJ     • 

rt 

0 

BJ 

S 

to 

to 

p 

p 

LO 

to 

•H 

■H 

bO 

bO 

O 

O 

t— 1 

rH 

o 

o 

^ 

.c 

o 

U 

X 

X 

10 

tO 

p< 

ft 

CM 


10    Pt 

to 

i 

(D    CD    aJ 

CD 

o 

r-l    4->    4-> 

— ( 

u 

CD 

^  a  nJ 

Q 

o^ 

> 

tH.HTl 

c3 

•H 

in 

•H 

H 

e 

P 

p 

sh  e  ^ 

Pi 

o 

U 

10 

aJ    O   CD 

aJ 

u 

CD 

o 

>    Pi  -H 

> 

m 

•r— i 

4-> 

<4H    > 

CO 


X 

G 

CCi 

PL, 

H 

H 

6 

U3" 

U5- 

o 

H 

H 

u 

< 

< 

r>N 

^ 

10 

> 

O 

>v 

r*               ^ 

rt  /-> 

P              LO 

P(   LO 

cS        lo 

PQ  lo 

^    Xto 

hO 

rt 

UET 

-^  i— •  11 

II 

4_)    CO 

P 

C        ^ 

P!  Z 

a)  uj ' — ' 

rt  ^-> 

U 

Jh 

CD 

CJ 

00 

O 

to 

CO 
0) 
LO 
CO 

rt 

<P 
O 

P< 

CD 

Pi 

Pi 
o 

m 

o 

•H 
P 
O 
CD 

Pi 
O 

o 


to 

CD 

P 


to 

CD 

X 

o 
PJ 
0 
p 
to 
■H 
to 

PJ 
o 
o 


PI 
Pi 
CD 
P 
P! 


cd 


to 


T3 
p. 

o 

a: 


CD 
O 

Pi 

3 

o 

CO 


89 


g.   Validity 

Byham's  article  [11]  provides  a  basic  primer  on 

validity  in  laymen's  terms.   At  the  other  end  of  the  spectrum, 

Taft  [61]  provides  a  wide-ranging,  probing  treatment  of  the 

subject.   With  respect  to  the  criterion  problem  he  writes: 

All  assessment  programs  involve  studies  of  the  link 
between  two  or  more  pieces  of  behavior,  whether  the  primary 
purpose  be  selection,  validation,  research  on  tests,  or 
personality  research.   Some  of  this  behavior. is   known  as 
assessment  behavior  and  some  as  criterion  behavior.   These 
concepts  are  analagous  to  the  independent  and  dependent 
variables  in  experimental  psychology,  and  it  is  an  arbi- 
trary decision  by  the  experimenter  which  one  is  designated 
which.   Most  of  the  reports  of  assessment  have  devoted  some 
space  to  the  criterion  problem  .  .  . 

A  special  problem  that  arises  in  personality  assessment 
is  the  frequent  unreliability  of  the  criterion  which  so 
often  represent  subjective  judgments  that  vary  from  one 
criterion  rater  to  another.   This  unreliability  imposes  a 
serious  limitation  on  the  potential  validity  of  personality 
assessments,  and  it  makes  it  difficult  to  evaluate  some  of 
the  low  validity  coefficients  reported  .  .  . 

The  assessment  strategy  should  be  aimed  at  the  criterion; 
once  the  latter  has  been  established  .  .  . 
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A  complication  that  arises  in  criteria  analysis 
is  that  an  assessor  can  only  predict  to  indices  of  the 
criteria,  not  to  the  actual  criteria,  themselves  .  .  . 

Sometimes  the  assessors  may  be  able  to  convince  those 
who  control  the  criterion  ratings  that  the  indices  which 
the  latter  are  using  are  not  consistent  with  their  funda- 
mental criterion,  but  eventually  the  assessors  and  the 
criterion  raters  must  agree  on  some  criterion  index  in 
accordance  with  the  policy  of  the  organization  .  .  . 

All  other  things  being  equal,  the  best  assessors  for 
predicting  existing  criteria  are  those  who  are  partially 
contaminated  with  the  same  experience,  standards,  and 
outlook  as  the  criterion  raters  and  can  thus  rely  on  a 
global  strategy  to  make  their  predictions. 

Taft  also  addresses  the  fact  that  criteria  are  dynamic  rather 

than  static: 
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A  further  condition  that  is  often  ignored  in  assessment 
is  that  of  effluxion  of  time;  the  predictions  are  usually 
made  on  the  assumption  that  the  status  of  the  candidate 
on  the  relevant  variables  will  remain  constant  over  time 


In  some  complex  situations,  in  which  the  criterion 
performance  is  highly  dependent  on  the  conditions ,  the 
inability  of  the  assessors  to  predict  the  specific  condi- 
tions that  will  operate  for  any  particular  candidate  may 
render  the  assessments  completely  invalid. 

This  latter  point  is  restated  and  amplified  by 

Ghiselli  and  Haire  [24]  who  write: 

When  we  validate  selection  tests,  we  typically  relate 
scores  to  some  measure  of  job  performance  obtained  during 
a  brief  initial  period  of  employment.   This  initial  period 
may  be  as  short  as  the  first  two  or  three  months  of  employ- 
ment, and  it  seldom  is  as  long  as  the  first  two  or  three 
years.   Presumably  the  men  are  hired  for  a  much  longer 
period.   The  rationale  of  our  validation  procedures  involves 
matching  a  predictor  variable  with  a  criterion.   But,  does 
the  brief  criterion  itself  predict  the  one  we  are  interested 
in- -performance  over  a  relatively  long  period?   To  be  sure, 
criterion  scores  for  the  initial  period  indicate  the  con- 
tribution workers  have  made  during  that  time,  but  this  is 
not  exactly  what  we  want  to  measure.   At  least  implicitly, 
early  criterion  data  are  taken  to  stand  for  ultimate  per- 
formance or  total  performance  on  the  job.   The  period 
selected  for  a  criterion  is  usually  determined  by  practical 
considerations  such  as  the  availability  of  data.   The 
question  to  which  this  paper  addresses  itself  is:   Do  tests 
validated  against  such  a  brief  initial  criterion  maintain 
their  validity  over  a  longer  period  of  measured  performance? 
There  are  indications  that  they  do  not  .  .  .  Worbois  [72]. 

The  practice  of  using  performance  data  obtained  during 
an  initial  period  and  letting  it  stand  for  ultimate  or  total 
performance  completely  ignores  the  dynamic  character  of  the 
criterion  and  important  changes  that  are  taking  place  in 
the  worker's  performance.   Over  a  considerable  length  of 
time  on  the  job,  there  is  not  only  a  change  in  average  per- 
formance, but  also  significant  changes  in  the  order  of 
individuals  in  their  performance.   The  change  in  average 
performance  may  be  much  more  than  we  are  used  to  thinking: 
we  are  used  to  laboratory  tasks  which  can  be  mastered  in 
an  hour  or  so.   On  the  j ob,  performance  may  improve  for 
many  months  or  even  years  .  .  . 

In  one  case,  the  authors  have  followed  the  progress  of 
a  group  of  investment  salesmen  for  ten  years.  During  this 
extended  period  there  was  650  percent  improvement  in  aver- 
age productivity  and  still  no  evidence  of  leveling  off. 

91 


Indeed  a  straight  line  would  fit  the  whole  data  well.   If 
there  are  significant  changes  in  order  within  such  a  period 
of  improvement,  early  validation  will  be  seriously  mis- 
leading.  A  good  many  studies  have  indicated  such  a  change 
in  order.   Contiguous  stages  of  learning  typically  correlate 
much  higher  than  periods  separated  by  some  time  .  .  .  Fac- 
torial analyses  of  learning  data  clearly  indicate  a  change 
in  weighting  of  factors  for  different  periods  of  the 
learning  process  .  .  .  All  of  these  things  suggest  that 
early  criterion  data  do  not  measure  all  of  the  important 
variables  in  performance. 

With  the  average  performance  and  rank  order  of  individuals 
both  changing,  one  is  tempted  to  turn  to  the  rate  of  improve- 
ment as  a  criterion.   If  ultimate  or  total  performance  is 
our  goal,  rate  may  be  a  useful  substitute.   To  wait  for 
final  performance  is  impractical;  even  maximal  performance 
may  be  so  far  off  that  it  delays  the  validation  of  new 
instruments  impossibly.   There  is  some  evidence  that  final 
level  is  predicted  by  rate  of  improvement,  so,  even  though 
it  indicates  little  about  an  individual's  level  of  perfor- 
mance during  the  initial  period,  it  may  be  more  indicative 
of  final  values  than  the  early  criterion  itself.   In  view 
of  the  difficulties  encountered  with  a  partial  criterion 
which  does  not  predict  the  ultimate  criterion,  this  study 
will  examine,  in  a  particular  case,  the  validation  of  tests 
for  the  prediction  of  differential  rates  of  improvement. 

From  their  study  of  56  taxicab  drivers,  Ghiselli  and  Haire 

arrived  at  the  following  conclusions: 

1.  It  is  suggested  that  the  dynamic  nature  of  the 
criterion  of  job  performance  presents  real  difficulties 
in  using  initial  measures.   If  the  desired  criterion  is 
ultimate  or  total  performance,  there  is  some  question 
whether  an  initial  criterion  measure  will  itself  be  a 
good  predictor.   If  not,  validation  against  this  early 
criterion  is  no  validation  at  all. 

2.  Data  are  presented  showing,  in  a  particular  case, 
job  performance  over  a  period  of  time.   During  this  time 
significant  changes  occurred  in  the  average  performance, 
the  variance,  and  the  rank  order  of  workers. 

3.  Validating  a  battery  of  tests  against  the  criterion 
data  obtained,  it  is  possible  to  predict  the  first  three 
weeks  of  performance,  or  the  last  three  weeks,  but  it  is 
not  possible  to  predict  both  with  the  same  battery.   The 
tests  show  various  patterns  of  validity:   low  validity  in 
initial  periods,  rising  later;  high  validity  in  initial 
periods,  falling  later;  and  cyclical  variations  in  valid- 
ity measured  against  successive  stages  in  performance. 
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4.   In  these  data,  it  is  possible  to  predict  rate  of 
improvement  with  some  success,  but,  over  the  brief  period 
investigated,  rate  is  not  strikingly  associated  with  total 
production. 

Bray  and  Moses'  [76]  review  of  the  literature 

produced  these  comments  regarding  the  criterion  problem: 

Such  heightened  concerns  about  test  validation  led 
naturally  to  the  "criterion  problem,"  a  recurring  problem 
through  the  years  because  few  researchers  directly  face 
up  to  this  issue.   A  little  progress  was  noted.   For  example, 
supervisory  ratings,  one  of  the  most  easily  obtained  cri- 
teria measures,  again  were  convincingly  discredited,  which 
no  doubt  will  discourage  few  from  using  them  in  the  future. 
An  implicit  point  also  under  fire  was  whether  criteria  used 
in  various  test  validation  studies  were  actually  pertinent 
to  the  test  validated. 

Another  important  issue  in  the  past  three  years  is  related 
to  criteria  for  selection  in  a  different  way.   This  is  the 
question  of  how  long  after  selection  a  criterion  measure 
should  be  taken.   This  concern,  as  well  as  a  somewhat 
newer  interest  in  developmental  changes  in  employees  has  led 
to  even  more  urgent  pleas  for  longitudinal  studies  than  was 
true  previously  .  .  . 

One  of  the  more  impressive  analyses  was  presented  by 
Ron an  and  Prien  .  ...  [123]  .  .  .  who  appear  to  be  moving 
us  closer  to  a  theory  of  criteria  .  .  .  Their  review  of 
the  literature  demonstrates  that  it  is  totally  unrealistic 
to  assume  that  job  performance  is  unidimensional .   A  single 
criterion,  even  a  composite  one,  is  often  very  misleading. 
.  .  .  many  investigators  have  not  been  attentive  to  the 
multidimensional  nature  of  job  performance.   Instead  they 
have  too  often  used  existing,  readily  available  organiza- 
tional indices  alleged  to  be  indicative  of  performance 
rather  than  devising  measures  focused  on  appropriate 
behavior.  .  . 

What  we  are  saying  is  that  some  very  serious  questions 
must  be  raised  about  the  efficacy  of  the  continued  and 
uncritical  use  of  supervisory  ratings  as  the  criterion 
measure.   Perhaps  this  is  why  we  have  a  criterion  "problem." 

Meyer  in  his  article:   "The  validity  of  the  in- 

basket  as  a  measure  of  managerial  performance"  provides  a 

description  of  test  design  and  validation.   He  also  performed 

a  cross  validation  in  his  study. 
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Guion  and  Gottier  [29]  provide  a  review  of  "Validity 

of  personality  measures  in  personnel  selection"  for  the  twelve 

year  period  1952  -  1963.   They  present  a  tabular  summary  of 

95  validation  studies  done  on  fifteen  different  personality 

and  interest  inventories.   Similar  summaries  are  included  on 

projective  measures,  special  inventories,  and  personal  history 

data.   Unfortunately,  their  conclusions  are  not  very  encouraging, 

as  they  note: 

There  is  no  .  .  .  evidence  in  the  survey  that  companies 
.  .  .  have  .  .  .  recognized  the  need  to  validate  the  instru- 
ments chosen  for  their  own  specific  situations  .  .  . 

Even  without  these  flaws  nearly  every  study  may  be 
criticized  as  following  the  traditional  paradigm  of  testing 
for  a  relationship  between  the  test  and  a  criterion.   Cer- 
tainly more  imaginative  designs  might  well  reflect  the 
undoubtedly  complex  relationship  between  motivational 
variables,  situational  variables,  ability  variables,  and 
behavior. 

They  also  echo  an  often-mentioned  bit  of  advice  for  the  program 

designer: 

Whatever  the  reason,  the  point  still  remains:   a  home- 
made personality  or  interest  measure,  carefully  and  com- 
pletely developed  for  a  specific  situation,  is  a  better 
bet  for  prediction  than  is  a  standard  personality  measure 
with  a  standard  system  of  scoring. 

This  review  of  references  concerned  with  validity 

is  concluded  by  Howard  [34] .   She  provides  tabular  summaries 

of  several  validity  studies,  as  follows: 

What  must  be  regarded  as  "The  Study"  in  assessment 
center  validity  is  ATQT ' s  Management  Progress  Study  .  .  . 
which  was  predictive  and  "uncontaminated; "  i.e.,  results 
were  retained  for  research  purposes  only  and  not  released 
to  management  to  influence  promotion  decisions.   The 
researchers  administered  the  assessment  procedure  to  422 
male  employees  of  six  Bell  Telephone  companies  beginning 
in  1956,  stored  the  results,  and  waited  eight  years  before 
pursuing  information  on  the  assessee's  progress  in  the 
company.   While  many  may  view  eight  years  of  waiting  as 
an  almost  unbelievable  display  of  forebearance ,  the  authors 
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admit  that  by  their  own  standards  they  were  impatient -- 
they  had  intended  to  wait  ten  years.   Comparisons  made  in 
1965  of  management  level  achieved  by  men  assessed  six  to 
eight  years  previously  are  shown  in  (Figure  9).   Validity 
for  the  assessment  predictions  was  amply  demonstrated. 


Figure  9. 

Relationship  Between  AT^T  Assessment  Staff  Prediction 
and  Management  Level  Achieved 

Prediction  if  make Status  in  July,  196  5    

middle  management     ~~%    1st  level  "  ~~%    2nd  level  \         \   middle 
within  10  years        management     management     management 


Yes  College 

(N  -  62)  2              50            48 

No  or  ? 

(N  =  63)  11              78            11 

Yes  Non-College 

(N  =  41)  7              61            32 

No  or  ? 

(N  =  103)  ■            60              35             5 

Note:   Adapted  from  Bray  and  Grant  [10].   Source:   Howard  [34] 

a  2  • 

x   significant  at  p  <^.001 

Point  biserial  correlations  were  .44  for  the  college 
group  and  .71  for  the  non-college  group.   Of  the  total  number 
of  men  who  reached  middle  management,  78  percent  were  correctly 
identified  by  the  assessment  staff.   In  contrast,  among  those 
in  both  groups  who  had  not  progressed  further  than  first 
level  management,  the  assessors  predicted  that  95  percent 
would  not  reach  middle  management  within  ten  years.   Note 
that  these  predictions  still  had  two  years  to  run;  later 
communications  from  the  company  indicate  that  even  greater 
accuracy  was  achieved. 

Correlations  between  assessment  ratings  of  general  effec- 
tiveness and  salary  increments  were  also  given  for  four 
samples  of  individuals  who  had  at  least  six  years  of  tenure 
in  management  since  being  assessed;  (a)  Company  A,  54  college 
men,  r  =  .41;  (b)  Company  C,  27  college  men,  r  =  .51;  (c) 
Company  B,  83  college  men,  r  =  .45;  (d)  Company  C,  39  non- 
college  men,  r  =  .52.   All  correlations  were  significant 
at  p  <  .01.   Combined  with  the  data  in  Table  2,  the  useful- 
ness of  AT&T's  overall  assessment  ratings  for  predicting 
management  success  seems  well  established . 


Another  ATfJT  study  with  newly  hired  candidates  for  sales 
positions  also  used  an  uncontaminated,  predictive  validity- 
paradigm  (5) .   The  primary  criterion  of  performance  was  a 
six-month  field  review  by  an  experienced  team  from  AT§T 
headquarters  which  regularly  makes  such  inspections.   Where 
the  assessment  judgment  was  "more  than  acceptable,"  100 
percent  of  the  salesmen  met  the  review  standards.   Comparable 
success  figures  for  those  judged "acceptable, "  "less  than 
acceptable,"  and  "unacceptable"  were  60  percent,  44  percent, 
and  10  percent,  respectively,  producing  a  chi-square  value 
of  24.19  (p  <  .001).   Again  the  predictive  validity  of 
AT&T's  overall  assessment  was  evident,  this  time  for  the 
job  of  salesman  and  over  a  short  time  interval. 

Problem:,  of  criterion  cont amina tion  have  confounded 
predictive  validity  studies  other  than  those  described 
above,  since  assessment  ratings  were  used  in  promotional 
decisions.   Where  assessment  ratings  were  used  primarily 
to  make  the  first  promotion,  the  effect  on  later  promotions 
was  not  felt  to  be  large,  however.   A  summary  of  these 
contaminated  criterion  studies  is  shown  in  Figure  10. 

From  the  studies  done  to  date,  overall  ratings  of 
potential  or  performance  from  assessment  center  procedures 
generally  have  shown  impressive  predictive  validity, 
especially  for  managerial  jobs.   Unfortunately,  use  of 
the  ratings  for  decision  making  about  assessees'  careers 
somewhat  restrains  an  overwhelming  acceptance  of  the 
findings.   Nevertheless,  "clean"  predictive  validity  has 
been  demonstrated,  but  only  in  two  studies  with  both  at 
the  same  company. 

h.   Peer  Ratings 

Kraut  [39]  reports  an  IBM  study  on  peer  ratings 

conducted  on  samples  of  managers  and  executives.   In  summary 

he  writes: 

In  the  business  world,  as  elsewhere,  people  regularly 
form  impressions  of  their  associates.   The  use  of  peer 
ratings  among  managers  may  merely  formalize  a  process  in 
which  managers  already  engage  informally.   This  study  indi- 
cates that  the  use  of  managers'  peer  ratings  is  likely  to 
raise  the  caliber  of  men  who  move  into  the  executive  ranks. 

A  source  which  deals  with  a  more  relevant  appli- 
cation is  that  by  Hollander  [33] .   His  paper  reports  a  study 
conducted  on  U.  S.  Naval  Officer  Candidates  circa  1955.   He 
reports : 
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The  evidence  presented  uniformly  supports  the  con- 
clusion that  peer  nominations  used  early  in  training  can 
make  a  distinctive  contribution  to  the  prediction  of  a 
long-range  criterion  of  performance  after  training.  A 
direct  implication  of  this  is  to  encourage  the  judicious 
use  of  such  early  evaluations  as  a  supplement  to  other 
predictive  measures. 

Another  military  application  of  the  use  of  peer 

nominations  (PNs)  is  reported  by  Amir,  et .  al .  [2] .   Their 

study  was  done  with  soldiers  of  the  Israel  Defense  Forces  in 

1964  -  1965.   A  total  of  3897  subjects  were  involved.   Their 

analysis  of  the  results  were  summarized  in  the  following 

paragraphs : 

What  explanation  can  be  found  for  the  high  PN  validity 
coefficients  reported  here  in  contrast  to  those  reported 
in  earlier  studies?   Moreover,  what  explains  the  higher 
coefficients  reported  for  promotion  to  officer  rank  from 
NCO  courses  than  those  predicting  promotion  to  NCO  courses 
from  basic  training?  ...  In  respect  to  the  former  question, 
the  following  reasons  appear  relevant:   The  present  inves- 
tigators observed  a  high  level  cf  task  orientation  displayed 
by  the  soldiers  who  responded  to  the  PN  questions.   Their 
attitude  may  be  a  function  of  the  knowledge  shared  by  most, 
if  not  by  all,  of  the  troops  that  their  present  or  future 
commanders  may  be  called  upon  at  any  moment  to  demonstrate 
their  leadership  abilities  in  actual  combat.   Therefore, 
the  men  tend  to  choose  those  of  their  companions  who  really 
impress  them  as  possessing  these  qualities.   Incidentially , 
this  high  level  of  task  orientation  may  be  inferred 
indirectly  from  the  lack  of  contamination  of  the  PN  scores 
with  friendship  choices.   The  validity  coefficient  of  PN 
scores  after  friendship  scores  are  partialled  out  was  .55 
as  compared  to  .58  for  basic  training  prior  to  the  partially 
out  of  the  friendship  score   and  .78  as  compared  to  .79 
for  NCO  courses. 

A  second  possible  explanation  for  the  relatively  high 
PN  coefficients  reported  here  lies  in  the  fact  that  soldiers 
are  wel]  acquainted  with  the  army's  criteria  for  promotion 
to  higher  rank.   This  information  appears  to  be  widely 
disseminated  among  the  troops  as  a  result  of  the  close  and 
informal  contact  maintained  between  soldiers  of  all  ranks. 
Such  informal  relationships  facilitate  communication  between 
ranks.   However,  these  reasons  appear  to  be  more  potent  for 
troops  in  NCO  courses  than  for  those  in  basic  training, 
which  is  perhaps  why  predictions  regarding  promotion  from 
NCO  to  officer  rank  are  of  greater  validity  than  those  from 
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basic  training  to  NCO  rank.   Troops  in  training  for  NCO 
rank  appear  to  identify  more  closely  with  the  values  and 
requirements  of  the  army  than  do  the  recruits,  and  are 
already  an  integral  part  of  the  military  leadership.   Con- 
sequently, they  have  more  understanding  of  the  qualities 
and  demands  of  leadership  than  do  newly  enlisted  troops, 
so  that  the  PN  choices  of  the  NCO  trainees  are  likely  to 
have  greater  predictive  power  than  those  of  troops  in  basic 
training . 

The  final  selection  chosen  relates  directly  to 
the  assessment  center  use  of  peer  ratings.   Finkle  and  Jones 
[85]  provide  a  discussion  on  the  use  of  such  ratings  from  a 
design  standpoint  and  cover  some  of  the  advantages  and  dis- 
advantages of  including  peer  ratings  in  an  assessment  program: 

As  in  situational  exercises,  we  expect  that  peer  ratings, 
if  they  are  to  be  used,  have  to  be  constructed  for  any 
assessment  program  in  accordance  with  the  format  and  objec- 
tives of  the  program.   The  peer,  or  sociometric,  ratings 
can  provide  the  assessment  committee  with  information  as 
to  how  the  assessees  themselves  judged  or  viewed  one  another. 
Such  information  can  be  solicited  at  the  end  of  the  program, 
at  times  during  the  program,  or  both.   Questionnaires  can 
be  designed  to  cover  opinions  about  particular  parts  of  the 
program,  such  as  a  group  exercise  just  completed,  or  about 
overall  skills  or  characteristics  witnessed  throughout  the 
program.   The  manner  of  questioning  can  be  as  varied  as 
any  form  of  rating  including  asking  assessees  to  rank  one 
another  (including  or  not  including  oneself)  on  overall 
contribution  to  an  exercise  or  on  specific  characteristics 
asking  them  to  write  out  opinions  or  explanations  about 
their  own  performance  or  that  of  others,  asking  them  to 
rate  all  other  assessees  on  scales  similar  (or  not)  in 
content  to  the  variables  the  assessment  committee  will  later 
employ,  or  asking  them  to  nominate  one  or  more  persons 
from  the  total  assessment  group  as  being  the  best  in  certain 
areas . 

Obviously,  the  perspective  of  the  assessees  will  not  be 
like  that  of  manager  observers  in  several  respects.   Most 
importantly,  they  will  simply  not  have  been  paying  as  much 
attention  to  one  another  as  to  themselves.   This  means  they 
really  have  a  limited  basis  for  rating  one  another.   Then, 
too,  their  judgment  as  to  the  significance  of  certain  actions 
and  abilities  may  be  less  realistic  than  that  of  experienced 
managers.   Finally,  peer  ratings  are  always  somewhat  suspect 
to  popularity  contests  (despite  some  clear  research  evidence 
to  the  contrary) - -though  absence  of  exposure  to  one  another 
prior  to  the  assessment  program  should  forestall  this  pos- 
sibility considerably. 
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i.   Assessors 

A  thorough  discussion  of  the  selection  and  training 

of  assessors  is  provided  by  Byham  [11]  who  writes: 

Choosing  the  assessors 

Typically,  assessors  are  line  managers  working  two  or 
three  levels  above  the  man  being  assessed.  .  .  .  These  are 
the  individuals  who  are  responsible  for  promotion  and  who 
know  most  thoroughly  the  job  requirements  of  the  positions 
one  level  above  the  candidate's. 

The  job  background  of  the  assessor,  of  course,  depends 
on  the  purpose  of  the  specific  assessment  center.   Where 
broader  management  aptitudes  are  being  assessed,  it  is 
common  for  the  assessors  to  be  drawn  from  a  number  of  areas 
in  a  company.   This  not  only  brings  in  a  number  of  viewpoints, 
but  exposes  the  candidate  to  representatives  of  a  number 
of  areas  where  he  may  find  promotional  opportunity.   Having 
representatives  of  different  areas  also  increases  the 
acceptance  of  the  findings  throughout  the  company. 

Assessors  from  management,  like  the  candidates  them- 
selves, are  usually  nominated  by  their  superiors  (although 
in  a  few  companies  the  center  administrator  makes  an 
effurt  to  recruit  thenij  .   Naturally,  the  practice  hr<s  its 
dangers.   After  a  center  has  passed  from  the  experimental 
to  the  operational  phase,  "purity"  controls  may  be  relaxed 
somewhat,  and  senior  management  may  be  tempted  to  send 
"cooperative"  managers  to  centers  to  act  as  assessors. 
This  temptation  is  particularly  strong  where  the  assessors 
serve  for  extended  terms. 

Center  administrators  have  chosen  to  react  to  this 
problem  in  various  ways.   Some  companies  rely  on  their 
assessor  training  programs  to  screen  out  assessors  who 
are  unacceptable  in  the  role,  for  one  reason  or  another. 
The  rationale  here  is  that  it  is  easy  to  spot  an  unquali- 
fied assessor  during  training  and  ease  him  out  without 
bloodshed.   As  a  fine  point  of  strategy,  for  example,  many 
center  administrators  suggest  that  it  is  wise  to  establish 
a  pool  of  assessors,  rather  than  train  asssessors  for 
specific  assignments.   With  the  pooling  arrangement,  it 
is  easy  for  the  administrator  to  bypass  unqualified 
assessors . 

A  major  point  of  controversy  among  operators  of  assess- 
ment centers  is  the  desirability  of  using  professional 
psychologists  rather  than  specially  trained  managers  as 
assessors.   Most  arguments  for  using  psychologists  are 
based  on  their  skills  in  observation;  they  are  trained 
to  recognize  behavior  not  obvious  to  the  untrained  eye. 
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While  this  argument  is  plausible,  it  has  yet  to  be  demon- 
strated in  an  operational  center.   Three  studies  have  found 
no  differences. 

However,  the  superiority  of  psychologists  over  completely 
untrained  managers  is  well  established.   Because  of  this 
superiority,  companies  often  use  psychologists  as  assessors 
in  experimental  or  pilot  programs,  where  training  management 
assessors  would  be  difficult.   Psychologists  are  also  used 
extensively  for  assessing  higher  levels  of  company  manage- 
ment; at  high  levels,  it  is  difficult  to  get  and  train 
managers  who  do  not  know  the  candidates  personally,  and 
the  objective,  independent  psychologist  is  seen  as  the 
fairest  evaluator. 

By  and  large,  companies  now  prefer  to  establish  a  pool 
of  trained  manager-assessors,  each  of  whom  serves  more  than 
once.   Individual  assessors  are  usually  drawn  from  the  pool 
to  serve  once  or  twice  a  year- -a  few  companies  ask  assessors 
to  serve  only  once.   AT&T's  practice  is  exceptional -- it 
assigns  assessors  for  six-month  terms  and  center  admini- 
strators for  one  year. 


Where  the  appointment  is  for  an  extended  period  of  six 
months  or  so,  of  course,  more  rigorous  and  lengthy  assessor 
training  is  feasible- -ATqT  trains  managers  for  a  month- - 
and  longer  experience  in  the  role  is  very  valuable  to  an 
assessor.   One   substantial  disadvantage  of  the  long  assign- 
ment is  that  assessment  becomes  a  routine  matter,  which  it 
never  should.   Reports  from  fatigued  assessors  read  like 
computer  output,  and  it  is  hard  to  think  of  them  as  anything 
more.   Currently,  only  AT$T  appoints  assessors  for  prolonged 
periods . 

Training  the  assessors 

In  the  companies  now  operating  assessment  centers,  there 
is  a  notable  difference  in  the  emphasis  placed  on  training 
assessors.   Some  companies  give  new  assessors  as  little  as 
one  hour  of  training,  which  really  amounts  to  just  an 
orientation  to  the  whole  procedure,  while  most  others  spend 
three  or  four  days. 
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One  can  argue  that  the  task  of  an  assessor  is  similar 
to  the  requirements  of  most  managers'  jobs- -a  manager  must 
interview  individuals,  observe  groups,  and  evaluate  presen- 
tations.  Assessing  requires  skill  in  these  same  areas, 
and  hence  many  feel  that  there  is  little  justification 
for  futher  training. 

The  principal  rebuttal  to  these  arguments  is  this: 
because  a  man  has  been  doing  something,  he  has  not  neces- 
sarily been  doing  it  well.   Companies  reported  marked 
improvements  in  the  reliability  of  supervisory  ratings 
after  the  supervisors  have  been  trained  to  work  as  assessors. 
Nonprofessionals  need  to  be   shown  what  to  look  for  in 
observing  group  discussions  and  individual,  presentations , 
or  they  may  focus  on  purely  surface  characteristics.   While 
rigid  scientific  studies  are  lacking,  it  is  obvious  from 
comparing  the  reports  presented  by  experienced  and  inexperi- 
enced assessors  that  training  makes  a  very  big  difference 
in  the  quality  of  performance. 

The  most  common  method  of  training  is  by  understudy. 
In  the  usual  situation,  an  assessor  -  in- training  sits  through 
an  entire  assessment  cycle  as  a  nonvoting  member.   Another 
method  of  assessor  training,  particularly  when  assessment 
centers  are  being  introduced,  is  to  have  the  assessors  go 
through  the  assessment  experience  first  as  candidates. 
Everything  is  the  same  except  that  there  are  no  assessors 
present.   In  a  typical  training  situation,  the  assessors 
go  through  an  activity  such  as  group  discussion  and  then 
critique  the  discussion  and  identify  possible  areas  of 
observation  afforded  by  the  situation.   Several  companies 
videotape  activities  to  give  assessors  practice  in  making 
observations . 

Byham  and  Pentecost  [12]  repeat  much  of  the 

foregoing  information,  and  add  the  following  points: 

Some  companies  believe  that  this  experience  is  so  bene- 
ficial that  they  have  gone  to  a  one-to-one  assessor-candidate 
ratio  in  order  to  expose  more  people  to  it. 

Almost  all  of  an  assessor's  training  and  experience  can 
be  applied  to  his  job  and  should  improve  his  ability  to  inter- 
view and  appraise  his  subordinates. 

Greenwood,  et.  al .  [28]  make  the  following  comments 

concerning  assessor  selection  and  training: 

.  .  .  the  instructions  to  the  evaluators,  whether  profes- 
sional or  nonprofessional,  should  be  explicit  as  to  the 
pecific  type  of  behavior  to  be  evaluated,  examples  provided 
s  to  kinds  of  behavior  that  may  be  expected  to  be  elicited 
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by  the  specific  situational  exercises,  and  standardized 
rating  forms  provided  for  their  use.   However,  the  results 
tend  to  indicate  that  the  selection  of  evaluators  for  this 
type  of  assessment  need  not  be  based  on  whether  they  have 
completed  an  extensive  training  program  or  have  professional 
experience  in  personnel  assessment. 

Taft  [61]  makes  a  strong  point  in  support  of  line 

managers  as  assessors: 

All  other  things  being  equal,  the  best  assessors  for 
predicting  existing  criteria  are  those  who  are  partially 
contaminated  with  the  same  experience,  standards,  and 
outlook  as  the  criterion  raters  and  can  thus  rely  on  a 
global  strategy  to  make  their  predictions. 

He  makes  two  more  pertinent  observations  in  his  summary,  e.g.: 

The  assessors  should  be  selected  for  proven  ability  to 
make  accurate  judgments  in  the  assessment  situation,  i.e., 
they  should  be  validated. 

The  assessors  should  be  familiar  with  the  criterion 
situation,  and  should  take  this  situation  into  account 
when  they  make  the  predictions. 

Concerning  the  alleged  "side  benefits"  claimed 

for  assessor  experience,  Howard  [34]  makes  the  following 

critical  observation: 

Training  Assessors  - -Benefits  of  assessor  training  have 
been  claimed  not  only  in  the  form  of  a  partial  solution  of 
the  criterion  problem  but  through  (a)  improvements  in 
interviewing  skills  (b)  broadening  of  observation  skills 
(c)  increased  appreciation  of  group  dynamics  and  leader- 
ship styles  (d)  new  insights  into  behavior  (e)  strengthening 
of  management  skills  through  working  with  simulations,  and 
(f)  broadening  one's  repertoire  of  responses  to  problems. 
No  well-designed  training  studies  have  validated  these 
promises,  however;  as  has  been  pointed  out  previously, 
firms  do  considerably  more  management  research  on  selection 
than  on  training  and  development  .  .  . 

In  summary,  Bender  [3]  compiled  the  following 

information  from  his  survey  of  assessment  centers: 
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Figure  11. 
Selection  and  Training  of  Assessors 

Number  of  organizational  levels  of  assessors  above  assessees:** 

Response  frequency   0   18   7   4 

Number  of  assessors  used  when  your  assessment  center  operates: 

3iJL_6Z£9111224 

Response  frequency   3   9   3   12   1   1   1    1    1    1 

Number  of  days  you  devote  to  training  one  assessor: 

5  hours   1   2   2_^_5   3   3^_5   4   5   6   9   10   15 
Response  frequency     3     431    5"  1  '   2   6   1   1    3   2 

**0ne  organization  indicated  that  only  psychologists  and  staff 
personnel  were  used  as  assessors. 

Source:   Bender  [3]. 

j .   Evaluation  of  Candidates 

Byham  and  Pentecost  [12]  introduce  the  topic,  with 

a  brief  overview  of  the  evaluation  process: 

The  professionalism  of  an  assessment  center  is  probably 
best  measured  by  forms  provided  to  assessors  to  record  and 
report  their  observations.   These  can  range  from  a  detailed 
description  of  all  pertinent  behavior  to  a  series  of  l-tc-5 
ratings  on  key  variables.   Well  -  developed  forms  can  guide 
the  assessors  in  making  observations  and  can  aid  them  in 
structuring  reports. 

There  are  several  ways  of  reaching  assessors'  consensus 
on  the  candidates,  among  them  a  method  based  on  predetermined 
factors.   Discussion  centers  on  one  candidate  at  a  time.   The 
assessor  reports  to  the  group  on  the  exercise  he  has  observed; 
all  assessors  then  rate  the  candidate  1  to  5  on  the  list  of 
factors.   This  procedure  is  repeated  for  each  exercise 
reported  and  the  assessors  then  rate  the  candidate's  overall 
performance  on  eacli  factor.   Next,  each  assesssor  announces 
his  rating  on  each  factor  and  differences  are  discussed 
until  consensus  is  reached. 
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The  advantage  of  the  factor-rating  approach  is  that  it 
forces  the  assessors  to  consider  as  important  the  whole 
range  of  behavior  identified.   Otherwise  the  assessment 
decision  might  fail  to  consider  important  areas  because 
of  overriding   positive  or  negative  performance  in  others. 
Also,  the  assessment  decisions  tend  to  be  more  organized. 

Another  approach  is  for  assessors  to  give  reports  on 
a  candidate,  one  after  another   while  all  assessors  take 
notes.   A  general  discussion  follows.   This  procedure 
facilitates  the  integration  and  interrelation  of  obser- 
vations, but  may  result  in  so  much  data  that  it  is  dif- 
ficult for  the  assessors  to  consider  all  of  it  at  one  time. 

Still  another  variation  is  to  have  one  assessor  write 
a  composite  report  on  a  candidate  from  individual  written 
reports  prepared  by  assessors  after  each  exercise.   The 
composite  reports  are  then  discussed  by  the  entire  group 
of  assessors  until  consensus  is  reached. 

A  more  thorough  treatment  is  provided  by  Finkle 

and  Jones  [85]  as  they  describe  the  procedure  used  in  the 

Sohio  program: 

Accepting  the  philosophy  that  we  learn  best  by  doing, 
we  begin  the  evaluation  of  our  first  assessee  after  having 
spent  approximately  an  hour  and  a  half  on  orientation. 
With  the  possible  exception  of  the  first  case,  the  assessees 
are  considered  in  alphabetical  order.   An  attempt  is  made 
by  the  staff  to  select  as  the  first  case  one  that  will 
probably  prove  to  be  rather  straightforward  and  consistent. 
We  do  not  feel  this  presort  is  critical  and  a  quick  scan 
may  not,  obviously,  accomplish  the  objective.   However, 
the  management  representatives  gain  further  insight  into 
the  variables  by  actually  putting  them  into  practice,  and 
we  feel  that  it  is  desirable  to  avoid  the  confusion  that 
may  arise  when  discussing   an  unusually  controversial 
individual.   Of  course  learning  can  and  does  occur  when 
there  is  a  good  deal  of  difference  of  opinion  which  arises 
because  of  the  complexity  of  the  assessee 's  personality. 
However,  understanding  how  to  use  the  variables  is  dif- 
ficult enough   in  the  first  case  without  adding  further 
complications  if  they  can  be  successfully  avoided. 

The  same  procedural  format  is  used  for  all  evaluations. 

1.   Pictures  of  the  assessee  are  circulated  among  the 
staff  members  to  refresh  memories  and  to  be  sure  that  those 
who  have  seen  the  assessees  in  operation  associate  the  name 
and  face  correctly.   A  rating  booklet  and  a  completed 
assessment  summary  sheet  containing  the  relevant  test  results 
is  given  to  each  staff  member  together  with  a  copy  of  the 
biographical  questionnaire  the  assesr.ee  filled  in  prior  to 
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the  program  and  a  set  of  note  paper  for  making  notes  on 
reports  presented  by  ether  members  of  the  assessment 
committee . 

2.  The  manager  who  conducted  the  personal  interview 
of  the  assessee  being  reviewed  presents  his  report  of 
information  which  complements  the  biographical  questionnaire. 
He  also  relates  the  general  manner  in  which  the  assessee 
handled  himself  during  the  interview. 

3.  The  psychologist  who  interpreted  the  projective  test 
materials  reads  his  report.   This  report  focuses  upon  the 
way  the  assessee  structures  and  organizes  his  work,  his 
attitudes  toward  people,  his  apparent  motivation,  and  his 
emotional  stability. 

(Note:   Programs  employing  projective  tests  are  in  the 
minority . ) 

4.  Then  the  report  of  the  assessee 's  performance  in 
each  of  the  situational  exercises  is  presented.   Each 
assessee  has  been  observed  by  a  different  management  repre- 
sentative at  each  exercise,  so  that  a  report  on  him  is  made 
by  each  of  the  management  staff.   Responses  of  the  assessee 
to  a  questionnaire  completed  after  eacli  exercise  are  read 
to  the  committee  by  the  program  administrator.   The  tables 
summarizing  the  peer  rankings  of  effectiveness  which  are 
completed  following  each  of  the  exercises  are  handed  out 
with  a  reminder  to  keep  them  through  the  review  of  all 
assessees  as  well  as  the  first. 

5.  A  discussion  is  held  of  the  appraisal  by  all  com- 
mittee members  of  five-minute  talks  that  were  given  by 
each  assessee. 

6.  The  report  forms  showing  the  results  of  a  final 
peer  questionanire  completed  by  the  assessees  as  their 
last  task  during  the  three-day  assessment  period  are  dis- 
tributed- -again  with  a  reminder  to  keep  them  through  all 
assessee  evaluations. 

During  the  presentation  of  information,  the  committee 
members  are  encouraged  to  ask  questions  as  each  report  is 
presented,  not  only  to  clarify  the  specific  information 
being  presented  but  also  to  add  further  knowledge  about 
the  procedure  and  the  nature  of  its  contribution  to  the 
program.   This  need  for  a  full  understanding  of  the  pro- 
cedures is  particularly  true  of  the  projective  test 
report.   Most  likely,  this  is  the  management  representatives' 
first  contact  with  the  concept  of  projective  testing.   It 
is  important  that  they  know  something  of  the  theory  under- 
lying the  use  and  the  limitations  of  this  type  of  testing. 
As  the  various  reports  are  fully  discussed  and  the  presenters 
questioned,  the  focus  of  this  discussion  should  be 
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clarification  rather  than  evaluation  as  such.   Either  the 
assessee  or  the  particular  procedure  in  use  should  be  better 
known  as  a  result  of  the  questioning.    The  professional 
staff  has  the  responsibility  to  see  that  the  committee  avoids 
premature  efforts  to  "type"  or  draw  inferences  rather  than 
give  or  digest  a  descriptive  report  of  findings. 

Once  the  personal  history  has  been  reviewed,  the  inter- 
view, projectives,  situational  exercises,  and  questionnaire 
information  presented,  and  the  five-minute  talk  discussed, 
the  assessment  committee  has  available  to  it  the  evidence 
upon  which  it  will  base  its  recommendations.   The  task  now 
is  to  draw  upon  this  information  to  judge  the  personal 
characteristics  in  the  rating  booklet. 

The  objective  is  to  build  a  consensus  of  opinion  con- 
cerning how  the  assessee  behaves  as  a  person  as  a  starting 
point  for  the  committee  and  other  users  of  the  information 
to  consider  the  implications  his  talents  and  style  have  for 
his  future  contribution  to  the  company.   The  committee 
members  are  instructed  to  rate  the  scales  on  the  basis  of 
their  judgment   after  considering  all  the  information. 
They  proceed  one  scale  at  a  time.   The  individual  members 
make  their  ratings  independently  and  announce  them  upon 
request  of  the  committee  chairman  after  all  have  had  full 
opportunity  to  arrive  at  their  scores.   Each  committee 
member  simply  calls  out,  upon  request,  the  scale  point 
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rating  booklet.   The  chairman  tallies  the  ratings  and 
calls  for  a  discussion  of  any  discrepancies  greater  than 
one  point  in  order  to  assure  that  all  committee  members 
are  weighing  all  relevant  information,  excluding  irrelevant 
information,  and  attaching  the  intended  meaning  to  the 
variable  being  rated  and  the  bench-marks  (scale  descriptions) 
on  the  scale.   If  the  review  of  sources  and  scale  meaning 
persuades  one  or  more  of  the  committee  to  change  his  mark, 
the  corrected  mark  is  made  in  red  and  the  committee  chair- 
man is  informed.   If  no  one  wishes  to  change  his  mark,  the 
discussion  is  closed  and  the  next  variable  is  considered. 
This  variance  in  committee  opinion  is  then  reflected  in 
the  narrative  report. 

Because  the  first  case  is  a  learning  exercise  as  well  as 
a  practical  handling  of  results,  it  is  well  to  avoid  the 
temptation  to  rush.   Time  is  taken  to  call  attention  to  the 
various  sources  of  information  throughout  the  protocol 
that  support  the  ratings  the  assessment  committee  has  arrived 
at.   Much  of  the  discussion  is  focused  on  the  scale  defini- 
tions so  that  the  concepts  will  become  better  crystallized 
for  the  management  representatives.   For,  as  understanding 
increases,  the  frequency  of  differences  in  ratings  which 
occur  primarily  because  of  differing  interpretations  of 
the  scales'  meanings  or  the  behavior  indicated  by  the 
scale  points  is  reduced. 
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We  have  found  that  1-1/2  to  2  hours  can  be  profitably 
spent  on  the  initial  case.   The  time  required  for  subsequent 
cases   falls  within  the  range  of  1  to  1-1/2  hours.   We  see 
no  evidence  that  a  reduced  amount  of  time  spent  on  a  particu- 
lar assessee  has  reduced  the  quality  of  assessment.   Our 
observations ,  admittedly  subjective,  lead  us  to  believe  that 
the  feeling  of  responsibility  of  the  assessment  committee 
members  causes  them  to  adjust  their  pace  appropriately  to 
the  complexity  and  needs  of  the  case  they  are  reviewing  and 
in  accordance  with  the  confidence  they  have  in  their  personal 
understanding  and  mastery  of  the  system. 

One  of  the  procedural  problems  we  have  been  alert  to  has 
been  the  possibility  that  the  interpretation'  and  integration 
of  the  cases  may  vary  as  a  function  of  the  sequence  in  which 
they  are  introduced  in  the  integration  session.   Yet  we  see 
no  evidence  that  having  an  assessee  discussed  early  or  late 
in  the  session  has  a  significant  bearing  on  the  quality  or 
utility  of  the  ratings  or  recommendations.   Repetition  of 
the  first  case  as  a  thirteenth   case  has  indicated  that 
being  first  did  not  affect  the  outcome.   Admittedly,  there 
have  been  times  when  one  or  more  members  of  the  professional 
staff  has  had  reservations  about  the  quality  of  the  committee's 
judgments  or  recommendations  with  regard  to  a  particular 
assessee.   However,  we  do  not  see  such  instances  as  related 
to  sequence  of  consideration,  to  length  of  time  devoted  to 
tne  case,  or  to  any  other  procedural  factor  that  might  be 
readily  adjusted,  improved,  or  controlled. 

Following  the  completion  of  ratings  on  the  various  scales., 
a  Viewgraph  transparency  is  used  to  project  on  a  screen  a 
summary  of  the  tallies  made  by  the  program  administrator  of 
the  committee  judgments   on  each  scale.   Also  on  this  slide 
is  shown--for  the  first  time--the  "pipe  organ"  diagrams 
showing  the  results  for  that  assessee  on  the  mental  ability 
tests.   When  these  scores  are  noted,  either  one  member  of 
the  professional  staff  or  the  program  administrator  reviews 
for  the  committee  the  ratings  given  on  each  of  the  scales. 
Reviewing  the  rating  scales,  of  course,  serves  to  consoli- 
date the  group's  thinking  about  the  assessee  so  that  all 
members  view  him  in  a  reasonably  comparable  way  before  con- 
sidering the  questions  of  placement,  development,  and 
potential.   We  have  adopted  the  practice  of  withholding 
information  about  the  assessee ' s  overall  mental  ability 
as  determined  directly  from  tests  until  this  time  to  avoid 
the  possible  biasing  effect  of  this  information  on  the 
judgments  of  other  characteristics  and  skills.   This  factor 
is  then  introduced  at  the  time  the  judgments   arc  reviewed 
and  is  considered  in  the  general  comments,  the  placement, 
supervisory,  development  and  potential  recommendations. 
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Resolving  Differences 

The  task  of  working  toward  a  resolution  of  those  dif- 
ferences of  opinion  that  inevitably  arise  in  the  rating  of 
the  behavioral  variables  warrants  discussion  before  we 
turn  to  the  process  for  developing  the  final  recommendations 
concerning  utilization  and  development.   In  actual  prac- 
tice, it  is  the  committee  chairman  who  exercises  the  key 
role  in  determining  whether  an  attempt  will  be  made  to 
resolve  differences  successfully,  at  least  to  the  satis- 
faction of  the  disagreeing  committee  members,  or  whether 
issues  will  be  merely  politely  debated.   However,  if  the 
debate  or  discussion  moves  into  a  somewhat  complex  con- 
sideration of  personality  theory,  human  behavior,  or  social 
interraction,  the  psychologists  present  will  naturally  be 
looked  to  as  the  experts  who  should  resolve  the  matter. 

We  mentioned  earlier  the  pressure  upon  the  professional 
staff  members  to  speak  competently  as  experts  at  times  and 
to  exercise  judiciously  their  responsibilities  to  aid  the 
management  representatives  in  the  interpretation  of  appar- 
ently contradictory  test  findings.   If  the  professionals 
are  not  careful,  however,  they  can  misperceive  their 
appropriately  felt  pressure  to  perform  in  a  professionally 
sound  manner  as  a  mandate  to  operate  infallibly.   Yet  we 
have  discovered  that  a  professional  staff  person  can  be 
quite  incorrect  in  his  reasoning  and-unduly  selective  in 
his  handling  of  the  information  available  about  the 
assessee.   It  is  of  course  expected  that  this  will  not 
occur  in  every  case.   The  greatest  danger,  however,  is  not 
whether  the  psychologist  will  be  wrong  in  some  instances- - 
he  will  be- -but  whether   he  is  willing  to  admit  his  error. 
If  he  cannot,  then  the  session  will  turn  into  little  more 
than  a  lecture  for  the  professional  staff  with  the  manage- 
ment representatives  and  other  committee  members  serving 
as  a  reluctant  audience.   Essentially,  then,  we  are  sug- 
gesting that  each  psychologist  must  be  willing  to  assume 
an  attitude  of  inquiry  and  participate  in  the  search  for 
the  best  possible  assessment  answers  rather  than  attempting 
to  promote  his  own  viewpoints  as  being  professionally 
correct. 

The  professional  staff  sets  the  tone  in  this  regard. 
Consequently,  if  the  professionals  operate  more  as  searchers 
than  advocates,  it  is  easier  for  the  management  representa- 
tives to  also  assume  this  approach.   Actually,  they  are 
inclined  to  do  so  anyway  and  need  only  the  support  and 
encouragement  that  the  psychologists'  approach  can  provide. 

Though  we  have  indicated  that  the  management  repre- 
sentatives look  to  the  professional  staff  for  leadership, 
this  is  not  to  suggest,  that  they  are  or  ought  to  be 
passive  followers.   Indeed,  our  experience  has  been  that 
the  management  representatives  have  been  quite  ready  to 


110 


contribute   their  own  ideas  and  have  been  quite  perceptive 
concerning  human  behavior.   In  short,  they  have  shown  little 
tendency  to  be  easily  cowed  or  accepting  of  prima  donnaish 
actions  on  the  part  of  the  permanent  staff.   Any  efforts 
toward  resolution  of  differences  have  clearly  been  joint 
efforts.   Happily,  the  focus  has  not  been  on  "who  is  right" 
but  rather  on  "what  is  right." 

Thus  a  difference  in  rating  or  recommendation  should  be 
approached  from  the  viewpoint  that  an  agreement  or  con- 
sensus may  be  desirable,  but  not  necessary  or  perhaps  in 
some  instances  not  possible.   Clearly,  no  one  should  change 
his  opinions  unless  the  available  evidence  leads  him  to  a 
different  interpretation.   Most  frequently,  any  disagreement 
in  ratings  occurs  as  a  result  of  attending  to  different 
portions  of  the  information,  and  the  difference  is  usually 
resolved  by  one  or  perhaps  both  of  the  disagreeing  members 
acknowledging  that  he  has  not  given  the  weight  to  a  specific 
piece  of  information  that  the  variable  under  consideration 
warrants . 

When  a  difference  does  become  apparent  as  the  committee 
members  announce  their  ratings,  each  of  the  differing 
views  is  given  a  hearing.   The  committee  member  who  has 
deviated  furthest  from  the  others  is  called  on  first  by 
the  committee  chairman.   The  committee  members  are  instructed 
to  view  this  opportunity  to  explain,  their  ratings  as  neithei 
a  sales  presentation  nor  a  defense  attorney's  plea.   Rather, 
each  should  present  the  evidence  for  his  rating  and  the 
reasoning  he  used  in  interpreting  the  evidence.   He  may 
question  the  other  committee  members  regarding  whether  they 
had  considered  the  same  evidence  or  if  they  can  point  up 
fallacies  in  his  logic.   Others  may  request  him  to  further 
amplify  his  evidence  or  reasoning,  or  if  it  seems  appro- 
priate, may  assist  him  to  better  understand  the  meaning 
of  the  variable  in  question  or  the  interpretation  of  the 
various  procedures.   As  indicated  earlier,  any  further 
discussion  that  moves  in  the  direction  of  persuasion  or 
disparagement  is  discouraged. 

The  chairman  should  also  keep  conversation  relevant 
by  sensing  when  a  discussion  has  reached  a  point  of  dimin- 
ishing returns  or  has  begun  to  stray  too  far  afield  in 
defense  or  rebuttal  of  a  point  of  view.   In  this  context 
it  is  important  that  the  professional  resist  the  temptation 
to  lecture  to  the  group.   We  have  already  mentioned  this 
in  pointing  out  the  danger  of  operating  as  though  the  task 
were  to  present  his  infallible  judgments.    Even  though 
the  professional  staff  member  assumes  the  spirit  of  inquiry 
we  have  described,  lie  is  in  some  danger  of  operating  too 
fully  in  an  instructor  role,  for  frequently  he  sees  an 
opportunity  to  enlighten  the  group  concerning  what  he 
feels  is  an  especially  valuable  psychological  insight. 
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We  do  not  suggest  that  he  always  refrain  from  following 
up  such  an  inclination,  but  we  do  recommend  that  he  do  so 
with  exceeding  caution  lest  he  sabotage  his  own  efforts 
by- becoming  irrelevant  or  technically  overbearing. 

Before  turning  to  a  consideration  of  the  development 
of  recommendations,  we  should  mention  that  most  of  the 
disagreement  in  any  integration  session  occurs  during  the 
rating  of  the  variables  rather  than  in  the  making  of 
recommendations  or  the  evaluation  of  potential.   We  feel 
that  this  is  as  it  should  be.   The  scales  have  demanded, 
in  effect,  that  the  committee  recognize  different  percep- 
tions, discuss  these  differences  openly,  and  clarify  their 
thinking.   By  the  time  the  developmental,  placement,  and 
other  recommendations  are  solicited  a  clear  and  more 
uniform  view  of  the  assessee  generally  prevails  among  the 
committee  members. 

Placement  Considerations 

In  considering  the  various  placements  that  might  be 
appropriate  for  the  assessee,  the  primary  criterion  is 
that  the  placement  be  one  that  provides  opportunity  for 
him  to  rely  heavily  upon  his  areas  of  strength.   It  is 
our  philosophical  conviction  that  both  the  individual 
and  the  corporation  are  best  served  by  placing  him  in 
areas  in  which  he  is  most  likely  to  experience  success. 
Common  sense,  our  own  experience  with  the  program,  and 
anecdotal  evidence  supplied  by  successful  business  leaders 
suggest  that  the  probability  of  success  is  greatest  when 
the  individual  is  employing  strengths  rather  than  trying 
to  respond  to  the  "challenge"  of  positions  requiring  him 
to  surmount  personal  liabilities.   Consequently,  any 
other  considerations  become  relevant  only  if  this  primary 
criterion  is  met. 

However,  the  committee  can  only  provide  suggestions 
since  a  number  of  factors  other  than  the  individual's 
strengths  are  also  considered  in  placement,  planning. 
Among  these  are  present  corporate  needs,  the  pattern  of 
future  corporate  growth,  availability  of  appropriate 
supervisory  climate,  and  the  broadening  exposure  provided 
by  the  assignment. 

Although  the  review  of  the  scale  ratings  has  pointed 
up  the  committee's  consensual  perception  of  the  assessee, 
it  is  basic  that  the  first  step  in  the  consideration  of 
placement  opportunities  be  to  gain  agreement  concerning 
which  of  the  persona)  qualities  should  be  capitalized 
upon.   Essentially,  this  entails  only  a  restatement  of 
the  assessee's  postivc  qualities  in  job-related  terms. 
For  example,  assume  the  individual  is  organized  in  his 
work  approach,  has  analytical  skill  and  extensive  finan- 
cial training,  but  has  limited  forcefulness  in  interpersonal 
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dealings  which  results  in  little  impact  on  others.   A  posi- 
tion requiring  integration  of  complex  financial  data  but 
with  little  or  no  responsibility  for  supervision  or  "selling" 
of  reports  would  most  likely  provide  best  utilization  and 
maximum  satisfaction  for  this  individual.   Of  course,  if  a 
position  could  be  found  that  also  provides  an  opportunity 
to  gain  greater  confidence   in  interpersonal  areas  through 
being  permitted  to  make  well -prepared  reports  to  a  group 
whicli  would  respect  the  assessee's  "expertise"  and  bear 
with  him  in  his  fumbling  but  courageous  speaking  efforts, 
so  much  the  better.   However,  this  would  be  a  secondary 
placement  target. 

Once  the  personal  strengths  to  be  utilized  have  been 
clearly  spelled  out,  the  committee  turns  to  the  task  of 
naming  several  positions  in  the  company  that  meet  these 
requirements.   These  positions  are  listed  not  as  recom- 
mended placements  per  se,  but  rather  as  illustrative  of 
the  type  of  opportunity  that  should  be  sought  for  the 
assessee.   Such  examples  help  clarify  for  later  users  of 
the  information  the  committee's  practical  interpretation 
of  all  of  the  rated  qualities.   It  is  desirable  to  identify 
positions  in  several  different  divisions  if  at  all  possible. 
Usually,  however,  the  three  management  representatives  on 
the  committee  at  a  particular  time  are  not  very  familiar 
with  all  segments  of  the  company.   This  further  supports 
the  concept  of  considering  the  committee's  placement 
suggestions  as  illustrative  only  rather  than  as  compre- 
hensive . 

This  is  a  part  of  the  assessment  process  in  which  the 
background  of  the  management  representatives  is  especially 
valuable.   They  are  asked  to  draw  upon  their  knowledge  of 
the  different  phases  of  the  business  to  identify  positions 
that  they  feel  make  the  type  of  demands  the  assessee's 
native  ability,  skills,  style,  and  experience  have  prepared 
him  to  meet  effectively.   These  positions  are  carefully 
considered  by  the  committee  with  the  professional  staff 
representatives  usually  taking  the  role  of  the  devil's 
advocate.   It  is  at  this  point  that  disagreement  is  most 
apt  to  occur  during  the  recommendation  preparation  phase. 
It  is  not  unusual  for  the  management  representatives  to 
mean  slightly  different  responsibilities  by  use  of  the 
same  term.   Areas  of  disagreement  are  resolved,  or  attempts 
are  made  to  resolve  them,  in  much  the  same  manner  followed 
during  the  scale  evaluation.   If  the  ambiguities  of  a 
particular  position  terminology  cannot  be  clarified,  however, 
such  a  position  should  not  contribute  to  the  subsequent 
confusion  of  others  by  being  used  as  an  example. 

•Occasionally  it  is  felt  by  an  assessment  committee 
that  no  current  corporate  positions  actually  are  illustra- 
tive oflxjw  best  to  employ  the  skills  of  a  particularly 
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unique  assessee.   In  these  instances  it  might  be  helpful  to 
suggest  the  design  of  a  new  position  and  the  way  in  which 
it  would  function. 

Also,  it  may  prove  constructive  in  some  cases  to  provide 
examples  of  positions  that  definitely  should  be  avoided 
because  the  likelihood  of  failure  would  be  quite  great. 
In  fact,  in  a  few  instances,  we  have  evaluated  individuals 
for  whom  it  seemed  less  important  that  specific  strengths 
be  utilized  than  that  some  very  pronounced  weaknesses  not 
be  employed.   Especially  for  these  assessees,  it  is  quite 
worthwhile  to  take  time  to  delineate  the  types  of  positions 
to  be  avoided  .  .  . 

Developmental  Needs 

When  introducing  the  area  of  development,  a  good  ground- 
work is  laid  by  subscribing  to  the  "self -development"  theory 
that  no  one  develops  a  person  but  the  person  himself.   Cor- 
porate management  is  responsible,  however,  for  providing 
the  climate  and  opportunities  that  will  enable  the  individual, 
if  he  so  chooses,  to  continue  his  development  in  the  areas 
into  which  his  experience  and  initiative  have  already  led 
him.   In  considering  what  types  of  opportunities  the  corpor- 
ation should  provide,  it  is  well  to  further  point  out  that 
those  qualities  that  are  already  strengths  should  be  further 
refined  if  possible.   Weaknesses  should  be  considered  only 
to  the  extent  that  they  are  significant  impediment:'  to  his 
performance.   Excessive  attention  devoted  to  those  liabili- 
ties that  do  not  seriously  limit  the  use  of  his  strengths, 
but  rather  detract  in  a  more-or-less  academic  way  from  his 
being  a  total  person,  may  not  be  either  a  constructive  or 
legitimate  focus  of  the  corporation.   Therefore,  in  evalu- 
ating the  desirability  of  public  speaking  training,  for 
example,  to  the  extent  that  such  exposure  would  increase 
an  already  moderate- to -high  level  of  competence  or  is  a 
necessary  skill  to  function  effectively  in  positions 
realistically  seen  as  options  for  the  assessee,  it  should 
be  encouraged.   However,  it  is  highly  questionable  that 
public  speaking  training  should  be  recommended  and  sup- 
ported by  the  corporation  if  it  might  not  prove  effective 
or  simply  because  it  might  assist  the  individual  in  gaining 
an  additional  skill  when  in  reality  it  would  be  unlikely 
that  this  additional  competence  would  materially  improve 
his  likely  contribution  to  the  company. 

We  believe  that  the  best  development  the  corporation 
can  provide  comes  from  job  placement  that  gives  opportunity 
to  apply  strengths.   Employing  strengths  increases  the 
likelihood  of  successful  accomplishments  which  in  turn 
improves  confidence  and  often  provides  greater  insights 
into  needs  for  broadening  and  development.   It  is  upon 
this  base  that  other  developmental  efforts  can  build  and 
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prove  effective.   Consequently,  some   of  the  most  important 
developmental  recommendations  are  contained  within  the 
section  on  placement  considerations. 

In  evaluating  further  developmental  needs  beyond  those 
satisfied  by  proper  job  placement,  the  committee  can  be 
given  a  listing  of  available  formal  and  informal  learning 
opportunities.   These  might  include  all  of  the  current 
training  programs  conducted  within  the  company  and  also 
examples  of  external  sources  and  activities  which  may  be 
appropriate.   It  is  not  unusual  for  the  committee  to 
decide  that  special  developmental  opportunities  beyond 
those  contained  within  the  recommendations  on  placement 
and  supervisory  climate  are  either  not  required  or  perhaps 
not  warranted  in  relationship  to  probable  return  on  the 
dollars  invested  in  providing  the  individual  with  a  partic- 
ular opportunity.   Return  on  investment  is  not  an  easy 
concept  to  measure  when  operating  in  the  often  intangible 
area  of  personal  development.   Nevertheless,  it  is  not 
without  merit  as  a  guiding  principle. 

A  wide  variety  of  specific  recommendations  may  be 
offered.   The  diversity  may  run  from  recommendations  of 
a  leave  of  absence  with  pay  to  permit  completion  of  a 
doctoral  program  for  a  talented  researcher  to  a  bibliography 
of  suggested  readings  in  management  practices  for  a  young 
supervisor.   The  overriding  concern  is  that  the  recommenda- 
tion will  enable  the  individual  to  be  more  productive  for 
the  company  if  he  chooses  to  take  advantage  of  the  suggested 
opportunity.   Here  again,  however,  the  recommendation  of 
the  committee  should  be  seen  as  suggestive  since  many 
factors  and  conditions  beyond  their  knowledge  and  control 
bear  on  the  advisability  and  timing  of  specific  develop- 
mental actions  by  or  on  behalf  of  the  assessee. 

Potential 

The  final  judgment  made  by  the  assessment  committee  of 
any  assessee  is  an  evaluation  of  his  potential  for  contri- 
buting to  the  company.   Again,  for  this  rating  a  5-point 
scale  with  half- step  intervals  is  employed.   Opportunities 
(numbers  of  jobs)  at  higher  levels  of  management  diminish 
quite  markedly  as  one  moves  upward.   Relatively  few  indi- 
viduals ever  move  into  serious  contention  for  officer 
level  positions.   And  yet  nearly  all  higher  level  positions 
are  filled  by  individuals  who-  move  from  the  levels  of  those 
now  being  assessed.   The  committee,  therefore,  should  try 
to  estimate  the  maximum  level  of  contribution  as  a  general 
guide  to  the  company  as  to  its  resource  potential  for 
filling  the  top  levels  of  responsibility.   In  actual  prac- 
tice, assessment  committee  judgments  of  potential  are  dis- 
tributed on  a  normal  curve  basis. 
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In  making  the  rating  on  potential,  the  objective  is  to 
estimate  how  far  in  the  organization  the  assessee  should 
advance  and  still  make  a  constructive  contribution  before 
the  normal  retirement  age.   This  may  be  different  from  the 
level  to  which  lie  will  actually  advance.   The  rating  on 
potential  may  be  higher  or  lower  than  the  level  he  will  in 
reality  attain.   Undoubtedly,  some  men  will  be  advanced 
in  the  organization  beyond  the  point  at  which  their  abili- 
ties enable  them  'to  be  effective.   Others,  through  circum- 
stances not  completely  within  their  control,  will  not 
advance  to  responsibility  levels  they  could  competently 
handle.   It  is  a  major  objective  of  the  assessment  program 
to  provide  evidence  and  judgments  that  will  improve  the 
likelihood  of  each  individual's  being  given  the  opportunity 
to  reach  his  maximum  level  of  contribution. 

The  assessment  program  will  not  be  without  error, 
however.   There  is  surprisingly  little  disagreement  at  this 
point  in  the  evaluation,  but  consensus  is  not  necessarily 
equated  with  accuracy.   We  simply  trust  that  the  careful 
deliberations  of  a  sincere  committee  will  provide  general 
guidance  that  can  be  constructively  used  in  subsequent 
manpower  deliberations . 

After  covering  current  operational  methods  of 

evaluation,  it  is  considered  prudent  to  review  some  theory 

on  how  the  process  might  be  improved.   Taft  [61]  provides 

this  function,  writing  as  follows: 

Let  us  conclude  this  section  on  the  safety  in  numbers 
assumption  with  a  proposal  to  combine  the  advantages  of 
both  multiple  techniques  and  multiple  assessors.   The 
suggestion  is  that  each  assessor  be  given  a  limited  amount 
of  information  on  which  to  base  his  assessment  judgments 
about  the  candidates,  each  assessor  to  receive  different 
information.   The  assessments  will  then  be  pooled  arith- 
metically.  The  information  supplied  may  be  objective 
or  subjective,  atomistic  or  molar,  and  may  range  from  one 
item  to  life-history,  or  a  test  result,  to  a  projective 
test  protocol,  an  interview  or  the  observation  of  behavior 
in  a  miniature  situation.   This  procedure  would  enable  a 
vast  amount  of  data  to  be  integrated  without  problems  of 
weighting  since  unit  weights  for  each  assessor's  contri- 
bution would  be  adequate- -  this  would  be  analogous  to  an 
inventory  that  gives  unit  weight  to  each  item.   With 
adequate  organization  of  the  assessment  program,  this 
would  permit  several  assessors  to  contribute  to  the  final 
assessment  so  that  different  viewpoints  and  personality 
theories  can  lie  represented.   This  approach  seems  to  be 
at  least  worth  experimenting  with. 
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Even  if  it  is  found  that  increased  numbers  of  assessors 
increases  (perceptibly)   the  accuracy  of  the  assessments, 
there  is  still  a  fine  calculus  of  cost  in  human  time  and 
effort  to  be  computed.   The  decision  to  augment  the  panel 
with  additional  assessors  is  a  function,  among  other  things, 
of  the  gradient  of  diminishing  returns,  the  ability  of 
available  extra  assessors,  the  cost  of  using  them,  the 
effects  on  the  candidates,  and  the  desire  to  allow  execu- 
tives in  the  institution  to  participate  in  the  assessment. 
The  proposal  made  above  of  having  many  assessors,  who 
contribute  small  pieces  of  information,  may  make  it  possible 
to  conduct  multiple  assessments  comparatively  cheaply. 

In  summary,  Taft  makes  the  following  recommendations: 

Each  assessor  should  be  given  no  more  than  two  or  three 
units  of  information;  there  should  be  a  large  number  of 
assessors  whose  predictions  are  pooled  arithmetically,  and 
without  discussion. 

In  selection  assessments ,  if  committee  decisions  are 
desired,  the  assessors  who  are  particularly  well -experienced 
in  the  criterion  situation  should  be  given  special  influence 
in  forming  the  final  decisions,  provided  they  have  been 
shown  to  possess  good  ability  to  judge  persons. 

k.   Feedback  of  Results  to  Candidates 

Once  again,  Byhams  [11]  treatment  provides  a 

brief  overview  of  the  topic.   He  writes: 

One  of  the  most  important,  yet  most  hazardous,  aspects 
of  assessment  center  operation  is  feeding  the  reports  back 
to  the  candidates.   Companies  handle  this  in  widely  dif- 
ferent ways,  depending  on  the  purpose  of  their  centers. 
Three  companies  offer  candidates  the  option  of  receiving 
or  not  receiving  feedback.   Between  60%  and  90 %  ask  for 
it.   These  companies  find  that  candidates  who  do  very 
well  and  those  who  do  very  poorly  usually  know  where  they 
stand  and  do  not  request  feedback,  whereas  those  in  the 
middle  want  to  find  out  how  they  did  and  get  hints  for 
self -  improvement .   Some  companies  give  feedback  to  all 
candidates  automatically. 

In  almost  all  cases,  and  certainly  in  companies  that 
are  strongly  concerned  about  management  development,  results 
are  carefully  couched  in  terms  of  the  directions  that  a 
candidate's  personal  development  should  take  in  the  future. 
The  candidate's  impact  on  his  fellow  candidates  may  be 
communicated  to  him  to  make  him  more  objective  about  himself, 
His  performance  on  individual  tasks  may  be  discussed  with 
an  eye  to  establishing  a  plan  to  overcome  noted  deficiencies 
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When  assessment  and  training  are  combined,  it  is  pos- 
sible to  provide  some  feedback  to  candidates  prior  to  their 
leaving  the  center.   In  some  companies,  a  candidate  must 
wait  weeks  for  a  feedback  interview.   Obviously,  the  sooner 
the  feedback  interview  takes  place,  the  more  impact  the 
training  and  development  recommendations  will  have. 

If  a  psychologist  is  available,  he  usually  has  the 
responsibility  of  discussing  the  center's  result  Kith  the 
candidate.   Otherwise,  assessors  or  former  assessors  are 
given  the  responsibility. 

The  USAIS  [125]  provides  the  following  counseling 

guidance  to  their  assessors: 

1.   The  following  is  an  outline  provided  to  facilitate 
counseling  feedback  based  on  assessment  exercises.   The 
collection,  collation  and  feedback  of  the  information  will 
take  place  as  follows: 

A.  The  Computer  Group  will  provide  each  counselor  a 
packet  of  information  relative  to  the  individual  he  is 
assigned  to  (counsel).  This  packet  will  contain  a  computer 
printout  of  all  rating  scales  arranged  by  dimension,  and 
the  narrative  portion  from  each  exercise.   A  summary  sheet 
containing  information  which  relates  to  each  assessee 
derived  from  paper  and  pencil  tests  will  also  be  included 
in  the  packet. 

B.  The  information  given  to  the  assessee  will  center 
around  the  dimensions  .  .  .  (specified) .   The  counselor 
is  charged  with  assimilating  the  information  from  the 
various  exercises  into  the  dimensions  for  counseling 
feedback . 

C.  The  counselor  will  prepare  his  counseling  inter- 
view in  the  following  steps: 

STEP  ONE:   Read  the  narrative  portions  from  each 
exercise  to  get  an  overall  impression  of  the  assessee. 

STEP  TWO:   Study  the  computer  printout  and  paper  and 
pencil  test  results. 

STEP  THREE:   Prepare  a  Narrative  Report  describing  the 
individual.   This  report  will  follow  the  dimensions  .  .  . 
(listed) .   The  report  will  contain  all  relevant  information 
from  the  narratives,  the  computer  printout,  and  the  paper 
and  pencil  test  summary  sheets.   This  report  will  serve 
as  the  basis  for  the  counsel iag  interview  and  for  ARI  two 
year  follow-up. 
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STEP  FOUR:   Prepare  the  counseling  interview.   The 
sequence  of  dimensions  covered  should  meet  the  following 
criteria: 

a.  The  first  dimension  selected  should  be  the  strongest 
of  the  assee's  relative  abilities. 

b.  The  initial  dimensions  should  be  the  easiest  to 
present  from  the  perspective  of  action  the  assessee  can 
take  to  address  his  weaknesses. 

c.  The  dimensions  in  which  the  assessee  is  weakest 
should  be  in  the  middle  of  the  sequence. 

d.  A  dimension  in  which  the  assessee  did  relatively 
well  should  be  reserved  to  conclude  the  interview. 

STEP  FIVE:   Review  and  select  tapes  to  support  the 
interview. 

(Notes:   (1)  The  14  leadership  dimensions  assessed  are: 
Social  Skills  (Interpersonal  Competence),  Communications 
Skills,  Adaptability,  Motivation,  Forcefulness ,  Mental 
Ability,  Decision  Making,  Administrative  Skills,  Organi- 
zational Identification,  Effectiveness  in  Organizational 
Leadership  Role,  Supervisory  Skills,  Physical  Competence, 
Technical  and  Tactical  Competence  and  Problem  Solving 
Ability. 

(2)  Step  Five  refers  to  the  preparation  of  a 
5-10  minute  videotape  which  the  counselor  records  from  the 
videotapes  of  the  leaderless  group  discussion  and  the  can- 
didate's interview  of  another  role-playing  candidate. 
Interviews  with  the  USAIS  assessors  disclosed  that  the 
edited  videotapes  were  a  powerful  counseling  tool  and  were 
often  able  to  dramatically  convince  candidates  who  had 
doubted  the  verbal  feedback  on  certain  specific  points.) 

Finkle  and  Jones  [85]  again  provide  a  thorough, 

although  specific,  description.   They  detail  the  feedback 

procedures  employed  in  the  Sohio  assessment  program  which  are 

somewhat  atypical  in  that  they  use  pscyhologists  to  conduct 

the  feedback  sessions,  where  most  programs  utilize  lay 

assessors.   In  spite  of  this,  the  description  contains  a 

great  deal  of  general  information  which  is  applicable  to 

any  feedback  system.   They  write: 
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.  .  .  two  basic  problems  quickly  arose  when  considering 
the  feasibility  of  a  feedback  session  .  .  . 

The  first  of  these  problems  concerns  the  cost  of  the 
feedback  in  both  time  and  money.   The  other,  and  even  more 
important,  relates  to  the  possible  misinterpretation  of 
the  results  by  the  assessee  and  the  consequent  damage  to 
confidence  and  desire  that  may  ensue. 

With  full  realization  of  the  problems  involved,  we 
decided  to  follow  through  with  a  feedback  session  for  any 
assessees  who  were  interested  in  requesting  one.   Weighed 
in  our  deliberations  were  the  following  considerations. 
The  cost  is  not  great  relative  to  the  investment  already 
made  in  the  program.   It  has  been  our  experience  that  an 
assessment  feedback  requires  at  the  maximum  an  hour  and  a 
half  time  and  more  typically  forty- five  minutes  to  an  hour. 
When  assessees  operate  in  areas  remote  from  the  home  office 
location  of  the  psychologist,  it  is  often  possible  to  arrange 
a  meeting  when  the  psychologist  is  at  a  nearby  company 
location  for  other  purposes  or  when  the  assessee  has  occasion 
to  visit  the  home  office  on  other  business. 

Furthermore,  the  cost  must  be  weighed  against  more 
intangible  outcomes.   It  is  impossible  to  calculate  the 
amount  of  resentment  toward  the  company  that  might  develop 
in  some  assessees  if  they  felt  that  pertinent  information 
concerning  their  personal  qualities  was  being  deliberately 
withheld  from  them  by  management.   The  loss  of  a  well- 
qualified  individual  as  a  result  of  such  antagonism  would 
be  a  real  loss  to  the  company.   Also,  the  cost  question 
must  be  evaluated  against  the  possible  gains  in  personal 
growth  that  the  feedback  might  foster.   We  will  return  to 
this  issue  a  little  later. 

The  possible  negative  effects  resulting  from  misin- 
terpretation of  test  findings  or  the  indelicate  handling 
of  the  reporting  of  the  findings  to  the  assessee  can 
adversely  affect  an  assessee  and  the  usefulness  of  the 
program.   Indeed,  even  though  the  feedback  is  conducted 
by  a  psychologist,  the  likelihood  of  misinterpretation  or 
improper  handling  is  only  lessened- -not  eliminated.   Yet 
there  is  no  assurance  that  distortion  and  misinterpretation 
do  not  also  arise  in  the  absence  of  a  feedback.   Obviously, 
each  of  the  assessees  speculates,  at  least  privately,  about 
his  own  abilities  vis  a  vis  the  other  participants  during 
the  various  group  exercises  in  which  he  is  involved.   How 
realistic  these  self -appraisals  are  depends  upon  a  number 
of  things,  including  the  maturity  and  insight  of  the 
assessee,  his  previous  experience  in  similar  situations, 
and  his  confidence  in  his  capabilities. 
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Without  the  opportunity  to  test  his  own  interpretations 
against  those  of  the  assessment  staff,  the  assessee  may  well 
err  either  in  the  direction  of  being  a  bit  too  optimistic  or 
pessimistic  concerning  his  future  prospects  with  the  company. 
In  extreme  cases  he  could  develop  a  totally  unrealistic 
outlook  regarding  his  potential  for  advancement.   Thus  the 
feedback  should  further  the  ability  of  the  assessee  to  better 
analyze  and  interpret  his  performance  and  the  possible 
implications  of  this  relative  to  his  career  aspirations. 

The  goals  of  an  effective  feedback  therefore  are  twofold: 
(1)  to  provide  the  assessee  with  an  opportunity  to  compare 
self -evaluation  of  his  performance  with  the  interpretation 
of  the  assessment  staff  and  (2)  to  provide  him  with  the 
appropriate  amount  of  feedback  information  in  a  growth - 
oriented  context  that  makes  the  information  as  acceptable 
and  valuable  to  him  as  possible. 

Initiating  Self -Development 

The  major  portion  of  the  writings  of  psychologists  and 
psychiatrists  supports  the  view  that  the  starting  point  in 
any  self -developmental  effort  is  a  willingness  on  the  part 
of  the  individual  to  examine  himself  and  determine  his 
current  abilities  as  carefully  as  possible  as  a  foundation 
for  formulating  personal  goals  involving  change.   To  be 
sure,  the  Lask  of  personal  growth  is  at  best  difficult 
to  govern  and  achieve.   For  in  addition  to  a  motivation 
to  better  fulfill  his  potential  capability,  an  individual 
needs  a  means  of  seeing  himself  as  clearly  as  possible  as 
he  is.   Actually,  the  assessment  program  offers  assistance 
in  this  respect   in  at  least  two  major  ways. 

First,  there  is  considerable  opportunity  during  the 
course  of  the  three  days  of  assessment,  to  critically  examine 
oneself.   On  the  measures  of  intellectual  ability,  the 
assessee  comes  to  grips  with  the  fact  that  he  can  handle 
some  types  of  intellectual  tasks  more  comfortably  than 
other  types.   Many  of  the  attitude  and  personality  inventory 
tests  force  him  to  examine  his  beliefs  and  his  personal- 
style.   Also,  the  personal  interview  confronts  the  assessee 
with  the  question  of  personal  goals  and  his  evaluation  of 
his  own  strengths  and  liabilities.   In  the  group  exercises 
the  assessee  has  the  opportunity  to  see  how  other  young 
businessmen  tackle  problems  and  conduct  themselves  in  a 
competitive  climate.   Finally,'  when  completing  the  final 
sociometric  questionnaire  at  the  conclusion  of  the  program, 
he  becomes  very  much  aware  that  not  only  is  he  evaluating 
others  in  terms  of  their  type  of  personal  impact,  but  that 
they  also  are  evaluating  him.   Naturally,  he  wonders  how 
he  impressed  the  others  during  the  three  days  of  formal 
and  informal  interaction. 
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For  many  of  the  employees  this  is  the  first  time  that 
they  have  really  stopped  and  taken  a  look  at  themselves 
in  depth.   Prior  examinations  have  typically  focused  on 
intellectual  -characteristics  only.   From  various  college 
quizzes  and  aptitude  tests,  they  have  gained  some  insight 
concerning  their  standing  relative  to  general  population 
or  student  norms.   There  may  have  been  some  vocational  or 
educational  counseling  at  the  time  of  the  selection  of  a 
college  and  career.   With  few  exceptions,  however,  they  have 
had  little  occasion  to  pull  back  and  reflect  upon  themselves 
as  individuals  in  a  broad  sense. 

The  second  means  by  which  the  asscssee  receives  assis- 
tance in  self-development  comes  in  his  opportunity  to  receive 
a  feedback  concerning  the  interpretation  of  his  performance 
and  a  chance,  to  discuss  himself  and  his  future  with  a  member 
of  the  assessment  staff.   Such  a  discussion,  if  well  handled, 
can  become  the  basis  for  more  sound  career  planning  and 
provide  both  a  clearer  focus  and  direction  to  personal 
developmental  efforts.   In  short,  the  feedback  session 
should  further  the  objective  of  personal  growth  for  those 
who  are  so  motivated. 

Personal  Feedback- -Not  Performance  Appraisal 

Because  a  performance  appraisal  interview  and  an  assess- 
ment feedback  both  deal  with  an  individual's  behavior  in  -: 
work  situation,  we  can  further  understand  the  nature  of  the 
assessment  feedback  by  comparing  the  two.   If  the  assessee 
has  had  performance  reviews  concerning  his  accomplishments 
on  the  job  at  fairly  regular  intervals,  he  will  expect  the 
assessment  feedback  session  to  be  rather  similar.   However, 
this  is  not  the  case  for  the  following  reasons. 

Basically,  there  is  a  pronounced  difference  in  the 
objectives  of  the  two  sessions.   The  goal  of  a  performance 
appraisal  discussion  is  to  provide  the  individual  with 
knowledge  about  bow  his  specific  job  performance  has  been 
regarded  for  a  stated  period  of  time,  usually  one  year. 
By  contrast,  the  assessment  feedback  deals  heavily  with 
the  effects  that  personal  characteristics  have  on  the 
assessee' s  present  and  potential  capacity  to  contribute 
productively  in  a  business  setting. 

It  is  widely  held  that  performance  appraisal  review 
sessions  have  their  most  constructive  effect,  when  the 
discussion  centers  about  the  accomplishment  or  lack  of 
progress  on  agreed-upon  work  objectives.   Discussion  can 
then  proceed  to  a  consideration  of  the  implications  this 
performance  has  for  the  attainment  of  future  work  objectives 
specific  to  the  individual's  current  job.   However,  when 
the  appraisal  session  moves  away  from  the  focus  on 
objective  results  into  an  examination  of  the  underlying 
behavioral  characteristics  that  made  the  performance 
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possible,  the  defensivcness  of  the  subordinate  increases  as 
the  ability  of  the  supervisor   to  evaluate  this  area  effec- 
tively comes  into  question.   Some  discussion  of  behavioral 
characteristics  is  perhaps  unavoidable,  but  the  temptation 
to  revamp  the  subordinate  as  a  person  is  difficult  to  resist 
and  there  is  danger  that  the  session  will  digress  noncon- 
structively  from  the  task  of  providing  knowledge  concerning 
job  progress  and  future  work  plans. 

By  contrast,  the  assessment  feedback  is  less  effective 
to  the  extent  that  it  becomes  too  related  to  specific  job 
performance.   Illustrative  examples  may  be  gleaned  from 
the  assessee's  current  job  performance,  but  the  emphasis 
is  clearly  upon  the  person  as  a  contributor  to  the  organi- 
zation in  general  rather  than  upon  his  behavior  in  a  given 
assignment. 

Another  factor  differentiating  the  feedback  from  the 
performance  appraisal  is  simply  who  conducts  it.   This 
distinction  arises  from  the  different  objectives  and  the 
relative  appropriateness  of  the  manager  to  deal  with  the 
information  to  be  covered.   Most  employees  accept  the 
concept  that  their  manager  is  a  legitimate  authority  to 
evaluate  the  results  of  performance  in  relation  to  agreed- 
upon  job  objectives  and  responsibilities.   Far  fewer  regard 
their  managers  as  holding  the  prerogative  to  either  assess 
or  attempt  to  change  basic  behavioral  characteristics  which 
in  turn  influence  the  attainment  of  work  results.   Therefore, 
when  the  manager  moves  into  these  areas,  the  subordinate's 
def ensiveness  increases. 

Essentially,  the  reverse  is  true  for  psychologists. 
Many  question  the  capacity  of  the  pscyhologist  as  an  evalu- 
ator  of  the  extent  of  accomplishment  of  specific  job  goals. 
However,  many  probably  agree  that  as  a  result  of  training 
and  experience  the  psychologist  is  likely  to  have  sug- 
gestions concerning  change  in  personal  qualities  that 
might  yield  dividends  in  improved  personal  satisfaction 
and  accomplishment. 

Also  the  psychologist  stands  in  a  different  power 
relationship  to  the  assessee  than  the  manager.   Although 
the  psychologist  may  be  recognized  as  a  representative 
of  the  company,  there  is  little  perception  of  the  psycholo- 
gist as  holding  any  immediate  real  power  over  the  assessee's 
future  as  is  the  case  with  his-  superior.   The  impact  of 
power  in  a  relationship  is  such  that  there  is  an  increase 
in  defensiveness  and  a  decrease  in  exploratory  and  creative 
thinking.   Thus  even  though  a  manager  may  have  a  positive 
superior-subordinate  relationship  with  the  assessee,  the 
inherent  power  structure  tends  to  have  a  dampening  effect 
upon  the  amount  of  personalized  information  that  might  be 
exchanged.   Basically,  then,  both  the  perceived  legitimacy 
of  the  psychologist  to  offer  suggestions  in  the  area  of 
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personal  growth  and  the  neutral  power  relationship  increase 
the  likelihood  that  the   assessment  feedback  will  become  a 
session  in  which  there  is  a  meaningful  focus  upon  personal 
qualities  and  formulation  of  growth-enhancing  plans. 

In  summary,  then,  the  differences  between  appraisal 
feedback  and  assessment  feedback  in  objectives  and  personnel 
involved  should  be  clear  to  those  involved.   However,  these 
two  activities  do  have  a  complementary  relationship  to  one 
another.   Both  activities  are  directed  toward  improving 
performance  and  providing  guidelines  for  further  develop- 
ment.  Ideally,  both  will  serve  as  vehicles  for  the  individual 
to  evaluate  better  his  current  performance  and  perhaps 
gain  further  insight  as  to  how  he  can  capitalize  most 
effectively  upon  the  stronger  aspects  of  his  performance 
and  adjust  other  attributes  if  his  capabilities  and  back- 
ground suggest  that  this  is  a  feasible  alternative. 

Here  again  the  manager  and  the  psychologist  both  work 
to  develop  and  to  utilize  better  the  individual  personnel 
resources  of  the  company.   The  manager's  emphasis  in 
working  with  the  individual  is  on  the  "what"  and  "how" 
aspects  of  his  results.   At  the  same  time  the  psychologist 
can  help  the  individual  better  understand  the  "why"  of 
his  behavior  and  gain  a  better  understanding  of  "where" 
he  is  apt  to  find  satisfying  application  of  his  talents. 

Start ing  the  Feedback  Session 

Let  us  turn  to  the  mechanics  of  the  feedback  and  two 
important  questions:   How  should  the  session  begin?   How 
much  information  should  be  conveyed?   These  questions  can 
be  best  handled  if  we  keep  constantly  before  us  the  pre- 
ceding discussion  and  remember  the  objectives  of  the  feed- 
back session.   In  actual  practice  the  conduct  of  the  feedback 
session  created  several  problems  for  us  early  in  the  program. 
A  large  portion  of  these  problems  arose  from  the  fact  that 
insufficient  attention  was  paid  to  the  context  of  the 
session . 

For  example,  our  early  feedback  sessions  were  too  much 
concerned  with  a  one-way  report  back  to  the  assessee  of 
major  assessment  findings.   The  criterion  for  a  successful 
feedback  thus  became  that  of  full  coverage  of  information, 
though  this  criterion  was  not  articulated  and  explicit. 
Consequently,  assessees  were  sometimes  given  information 
that  was  perhaps  interesting  but  of  questionable  relevance 
to  their  own  perception  of  their  career  objectives  a7id 
personal  needs.   Had  we  been  more  fully  cognizant  of  the 
full  role  the  feedback  session  can  play,  we  would  undoubtedly 
have  been  quicker  to  see  the  joint  problem- solving  nature 
of  the  feedback  session. 
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Another  difficulty  of  our  own  making  was  to  approach 
the  feedback  task  with  the  rather  naive  assumption  that 
all  assessees  aspire  to  positions  in  top  management.   The 
feedback  information  was  presented  to  the  assessee  in  the 
perspective  of  the  assets  necessary  to  progress  to  top 
management.   As  a  result  we  were  often  placed  in  a  position 
of  indicating  to  someone  who  had  modest  aspirations  in  the 
first  place  that  his  prospects  for  advancement  to  positions 
of  heavy  responsibility  were  quite  slim  because  of  the  lack 
of  relevant  personal  qualities.   With  the  advantage  of 
hindsight,  we  now  recognize  that  the  position  we  had  worked 
ourselves  into  was  not  unlike  that  of  a  talent  scout  for 
the  Metropolitan  Opera  telling  an  average  church  choir 
member  that  he  had  little  prospect  of  success  as  the  male 
lead  at  the  Met.   Even  though  the  choir  member  had  no 
operatic  ambitions,  he  would  be  a  bit  chagrined  at  the 
appraisal  of  the  scout.   Likewise,  our  assessees  who  had 
neither  the  aspiration  nor  ability  for  top  management 
responsibility  were  nevertheless  discouraged  and  resentful 
on  receiving  information  couched  in  a  top  management  frame 
of  reference. 

In  these  early  feedback  sessions,  it  was  particularly 
painful  to  anticipate  and  conduct  a  feedback  with  someone 
who  had  performed  poorly  throughout  the  assessment  session. 
However,  when  we  changed  the  feedback  from  a  simple 
reporting  of  results  to  a  problem-solving  effort,  even 
these  sessions  became  manageable  and  productive. 

Fortunately,  even  those  who  had  done  poorly  in  the 
assessment  session  either  had  an  awareness  of  this  and 
had  adjusted  their  aspiration  level  accordingly  or  did 
not  have  unrealistically  high  ambitions  initially.   We 
have  found  that  when  an  individual  is  given  an  opportunity 
to  describe  what  it  is  that  he  wants  from  his  business 
career,  very  few  men  express  aspirations  higher  than  the 
assessment  staff  felt  they  were  likely  to  attain.   Once 
the  assessee  outlines  his  objectives,  it  becomes  possible 
to  relate  his  performance  to  these  objectives.   Also,  by 
obtaining  very  active  involvement  from  him  early  in  the 
session,  it  is  easier  to  maintain  the  session  as  a  joint 
effort  rather  than  to  try  to  introduce  artificially  a  two- 
way  information  exchange  such  as  had  characterized  our 
early  efforts . 

Quite  often  an  assessee  states  that  he  wants  to  progress 
as  far  as  his  abilities  will  permit,  which  is  perhaps  a 
subtle  way  of  asking  for  the  assessment  staff's  rating  of 
his  potential.   In  discussing  potential  it  is  important  to 
stress  that  his  potential  depends  upon  his  own  talents  and 
efforts,  upon  his  exposure  to  jobs  and  people  in  order  to 
best  use  and  prove  his  abilities,  and  upon  environmental 
conditions,  including  the  need  for  his  talents,  the 
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availability  of  similar  talent,  and  a  grain  or  two  of  good 
luck.   We  have  found  it  most  effective  to  concentrate  on 
talking  about  the  relative  strengths  a:id  weaknesses  he  has 
and  discussing,  in  general,  how  he  can  continue  to  grow 
in  effectiveness  as  a  person. 

How  Much  Information 

The  question  of  how  much  information  to  provide  is  also 
a  thorny  issue  which  is  best  resolved  in  each  specific  case 
by  remaining  alert  to  the  counseling  context  of  the  feedback 
and  the  stated  goals  of  the  assessee.   The  assessment  findings 
should  be  evaluated  and  integrated  in  such  a  way  as  to  meet 
the  assessee '  s  needs  rather  than  simply  be  presented  in  a 
"take- it-or-leave- it"  fashion  without  such  integration  and 
interpretation.   Little  is  gained  by  giving  him  the  "straight 
scoop"  without  an  attempt  to  relate  it  to  his  individual 
desires  and  experience.   Specific  performance  on  a  given 
test  or  exercise  may  be  used  to  illustrate  a  broader  char- 
acteristic, but  the  overriding  judgments  of  the  committee 
are  the  principal  subject  for  review. 

Overall,  it  is  valuable  to  keep  the  discussion  focused 
on  a  descriptive  level  which  is  more  integrative  than 
evaluative.   For  example,  "You  seem  not  to  take  full  advan- 
tage of  opportunities  to  tune  in  to  the  needs  of  others  in 
your  thinking  and  action."   Inference  is  made  without  cor. 
demnation.   An  idea  is  offered  for  discussion  and  as  much 
as  possible  the  session  is  oriented  about  what  the  assessee 
is  ready  and  willing  to  accept. 

As  long  as  the  session  is  directed  toward  description 
and  interpretation  in  the  interest  of  self -understanding . 
there  is  less  likelihood  of  distortion  and  difficulty.   Nor 
is  there  danger  in  discussing  the  implications  of  present 
characteristics  for  the  type  of  person  the  assessee  hopes 
to  become  and  possible  steps  to  take  or  questions  to  ask 
himself  as  he  moves  toward  his  objectives.   However,  danger 
does  exist  in  trying  to  discuss  areas  that  are  essentially 
beyond  the  control  and  management  of  the  assessee  himself 
or  the  psychologist  with  whom  he  is  working. 

One  of  these  areas  is  the  question  of  long-range  potential 
as  evaluated  by  the  assessment  staff.   We  have  already 
expressed  our  feeling  that,  the  information  can  be  most  often 
misinterpreted  because  it  is  abstract,  future -oriented,  and 
subject  to  so  many  variables  outside  the  assessee  himself. 
Yet  one  of  the  basic  questions  the  assessee  wants  answered 
when  he  requests  a  feedback  is  how  much  potential  for 
advancement  he  is  regarded  as  having.   His  question  probably 
stems  from  a  need  for  the  security  of  reassurance  that  his 
fate  is  not  sealed  and  his  value  to  the  company  short-lived. 
Actually,  this  reassurance  can  be  given  to  most  assessees 
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without  the  necessity  of  delving  with  great  precision  into 
the  estimate  of  potential.   The  selection  criteria  for  the 
program  are  such  that  few  if  any  problem  cases  are  evaluated. 
Therefore  it  is  almost  always  possible  to  say  to  the  assessee 
with  sincerity  that  he  has  potential  for  growth  within  the 
organization.   This  fact,  combined  with  the  general  realism 
of  the  expressed  objectives  by  the  assessee,  places  the 
question  of  potential  in  proper  perspective.   Most  important, 
no  assessee  should  be  left  with  the  feeling  that  a  ceiling 
has  been  established  for  him.   He  should  recognize  that  any 
estimate  of  potential  is  just  that- -an  estimate.   Many  cir- 
cumstances, foremost  of  which  is  the  assessee 's  own  appli- 
cation and  development  of  his  abilities,  can  affect  his 
potential . 

Beyond  the  general  question  of  potential,  there  are 
other  limitations  that  it  may  be  desirable  to  place  upon 
reporting  the  data  to  the  assessee.   Thus  specific  jobs  as 
opposed  to  job  families  should  not  be  discussed  with  the 
individual  as  possible  future  placements.   It  should  be 
remembered  that  specific  job  titles  are  developed  in  the 
integration  session  simply  to  exemplify  better  the  type 
of  position  that  would  capitalize  upon  the  assessee's 
strengths.   These  are  examples  only.   When  discussed  with 
the  assessee,  special  care  must  be  taken  to  make  sure  the 
individual  understands  the  illustrative  use  of  the  position. 
Otherwise,  the  individual  is  apt  to  begin  thinking  in  terms 
of  a  specific  job  and  be  disappointed  if  it  is  not  soon 
offered  even  though  a  future  proffered  position  may  actually 
meet  his  needs  and  the  company's  needs  more  effectively. 

Likewise,  to  the  extent  that  the  areas  of  supervisory 
climate  and  training  are  discussed,  it  is  desirable  to 
discuss  only  those  things  over  which  the  assessee  has 
control.   In  speaking  of  his  relationships  with  authority, 
it  is  well  to  avoid  any  discussion  of  his  immediate  superior 
unless  the  assessee  brings  up  the  topic.   Frequently,  the 
psychologist  conducting  the  feedback  has  information  con- 
cerning the  superior.   Discreet  use  of  this  knowledge  may 
be  helpful  in  aiding  the  assessee  in  his  relationship  with 
the  superior,  but  most  often  the  psychologist  will  do  well 
to  remember  that  the  focus  of  the  feedback  is  the  assessee- - 
not  his  supervisor.   Frequently,  the  assessment  staff 
recommends  outside  advanced  management  programs  as  company- 
sponsored  training  activities  for  the  assessee.   It  may  be 
useful  to  advise  the  assessee  to  keep  himself  open  and 
receptive  to  training  opportunities  his  management  provides, 
but  beyond  this  there  is  nothing  to  be  gained  by  discussing 
specific  programs  where  the  assessee  does  not  control  the 
selection.   However,  any  relevant  activities  the  assessee 
can  initiate  fully  on  his  own  should  be  openly  and  fully 
explored . 
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Feedback- -Oral  or  Written  Report 

Our  feedback  sessions  have  been  completely  oral.   This 
has  provided  us  with  desirable  flexibility  and  lias  permitted 
a  full  "back-and-f orth"  discussion  with  eacli  assessee  to 
assure  better  understanding.   However,  even  with  oral  feed- 
backs there  is  the  danger  that  misperceptions  may  arise  and 
go  completely  unnoticed  by  the  psychologists.   There  is  also 
the  disadvantage  (at  least  in  the  eyes  of  the  assessee) 
that  he  has  nothing  tangible  to  carry  away  with  him  from 
the  session. 

Though  we  have  not  employed  a  written  report  for  assessees 
at  Sohio,  such  an  approach  could  of  course  be  considered. 
Presumably  it  should  be  descriptive  in  nature  and  similar 
to  the  one  provided  for  management. 

An  extension  of  the  arguments  favoring  a  written  report 
for  the  assessee  would  lead  to  the  conviction  that  he  should 
simply  be  given  a  copy  of  the  report  prepared  for  management. 
Another  view  might  be  that  he  should  be  given  all  detailed 
information  in  terms  of  test  scores,  interview  reports, 
exercise  reports,  and  so  on  that  were  used  by  the  integration 
committee  in  forming  its  judgments. 

Our  objective  in  establishing  the  most  effective  feedback 
approach  lias  been  to  maximize  understanding  en  the  part  of 
the  assessee  as  to  what  has  been  concluded  about  him,  how 
it  will  be  used,  and  how  it  is  likely  to  affect  his  career 
with  the  company.   We  doubt  that  an  open  presentation  (even 
"explaining")  of  "raw"  data  best  serves  this  purpose  in  the 
light  of  the  complexities  of  the  techniques  and  exposures 
used  to  obtain  the  data  and  in  view  of  the  nature  of  the 
multiple  judgments  contributing  to  the  conclusions.   A 
A^ritten  report  may  be  justified  but  not  if  it  cuts  off 
discussion  that  gives  the  psychologist  himself  feedback  as 
to  the  progressive  understanding  of  the  assessee  in  the 
feedback  process.   Oral  feedback,  when  well  handled,  seems 
to  do  the  best  job  of  developing  full  and  accurate  under- 
standing. 

It  should  be  remembered,  however,  that  we  are  seeking 
to  provide  the  assessee  with  an  understanding  of  the  signif- 
icance of  the  information  for  his  career  as  well  as  of  the 
content  of  the  information  itself.   Perhaps  the  best  indica- 
tor of  this  understanding  is  a  reading  by  the  psychologist 
of  the  assessee 's  emotional  reaction  to  the  feedback  session. 
Signs  of  disappointment  may  often  indicate  a  misunderstanding 
either  of  the  information  or  its  significance.   for  persons 
who  have  an  excessively  unrealistically  high  level  of 
aspiration,  discouragement  is  to  be  expected.   For  most 
individuals,  however,  the  overriding  reaction  to  the  feedback 
is  constructively  accepted.   Our  experience  suggests  that 
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the  persons  most  likely  to  feel  disappointment  are  neither 
those  with  exceptionally  great  talent  and  potential  nor  those 
with  relatively  modest  or  narrow  abilities  but,  rather,  those 
with  skills,  abilities,  and  interests  that  are  about  average 
or  a  little  better  than  average  for  the  population  of 
individuals  recommended  for  the  program.   This  is,  perhaps, 
somewhat  like  the  reactions  of  "B"-level  students  in  school. 
They  experience  neither  the  relief  of  the  "C"-level  student, 
who  may  have  had  some  doubts  about  passing,  nor  the  satis- 
faction of  the  "A" -student  who,  though  he  may  have  expected 
it,  is  rewarded  with  the  highest  possible  mark. 

There  are  several  ways  to  help  this  "level"  of  person 
achieve  an  appropriate  understanding  of  the  assessment 
information  and  its  significance  for  him.   The  psychologist 
must  remember  that  the  results  of  the  program  confirm  that 
the  assessee  is  exceptional  in  ability  and  potential.   Those 
selected  by  management  to  attend  were  seen  as  more  talented 
and  as  having  greater  potential  than  their  peers  in  the 
work  setting.   The  assessees  we  are  now  discussing  were 
seen  by  the  assessment  committee  as  having  as  much  or  more 
talent  and  potential  than  the  typical,  well -recommended 
assessee.   Clearly,  the  psychologist  should  focus  on  the 
confirmation  of  prejudged  ability  rather  than  on  the  point 
that  some  have  been  seen  as  having  even  more  talent  and 
potential   than  the  assessee  being  addressed.   Beyond  this, 
it  is  important  to  reiterate  what  should  have  been  told 
the  assessee  prior  to  the  program  and  what  certainly  has 
been  pointed  out  more  than  once  at  the  program,  namely, 
that  the  primary  objective  is  to  establish  and  reflect  on 
the  individual's  strengths  so  that,  through  his  greater 
awareness  of  these  strengths  and  the  company's  greater 
awareness  of  them,  they  can  be  more  quickly  and  more 
appropriately  applied.   Finally,  it  should  be  emphasized 
that  an  individual's  potential  depends  upon  opportunity, 
exposure,  performance,  and  upon  the  judgment  of  one  or 
more  key  people  who  have  the  authority  to  affect  the 
individual's  progress.   The  assessment  program  operates 
primarily  to  improve  exposure.   The  contribution  the 
individual  makes  in  the  eyes  of  those  he  is  working  for 
and  the  opportunity  those  persons  have  to  increase  his 
responsibility  are  still  the  primary  determiners  of  his 
career  advancement.   Each  person's  future  depends  upon  the 
extent  to  which  someone  who  feels  that  person  can  make  a 
stronger  contribution  in  the  organization  is  in  a  position 
to  do  something  about  it. 

Because  of  the  critical  importance  of  properly 

designed  and  executed  feedback  sessions,  it  is  considered 

important  to  close  this  section  with  some  additional  words 

of  caution  concerning  possible  damaging  effects  which  can 
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result.   Meyer  et .  al .  [49]  report  the  following  findings 

from  a  study  done  at  General  Electric: 

The  average  G.  E.  employee's  self -estimate  of  perform- 
ance before  appraisal  placed  him  at  the  77  percentile. 

'Only  2  out  of  92  participants  in  the  study  estimated 
their  performance  to  be  below  average. 

Commenting  further  on  these  findings,  Thompson  and  Dalton 

[63]  write: 

.  .  .  The  probability  is,  therefore,  that  for  701  to  80% 
of  all  technical  personnel  a  comparative  ranking  would 
be  a  deflating  experience.   Certainly,  all  would  feel 
criticized,  and  those  who  asked  for  the  reasons  for  their 
unexpectedly  low  rating  would  receive  more  specific 
criticism.   Judging  from  the  G.  E.  findings,  this  would 
have  a  disruptive  effect  on  the  performance  of  most  of 
them.   In  addition,  it  would  have  a  negative  effect  on 
their  self-esteem. 

The  message  voiced  repeatedly  in  these,  and  other, 

articles  is  that  feedback  of  results  is  one  of  the  more 

difficult  and  crucial  factors  in  the  design  of  an  assessment. 

program.   Thompson  and  Dalton  [63]  express  it  well: 

Performance  appraisal  touches  on  one  of  the  most  emotionally 
charged  activities  in  business  life- -the  assessment  of  a 
man's  contribution  and  ability.   The  signals  he  receives 
about  this  assessment  have  a  strong  impact  on  his  self- 
esteem  and  on  his  subsequent  performance. 

Although  the  comment  concerned  performance  appraisal,  this 
writer  considers  it  to  be  equally  applicable  to  assessment 
center  results. 

6 .   The  Rig  Picture:   Integrating  the  Subsystems 

The  need  for  an  integrated  systems  approach  to  per- 
sonnel management  was  mentioned  in  Section  I  of  this  paper. 
The  literature  supporting  that  need  will  now  be  reviewed. 
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The  basic  problem  is  stated  thus  by  Bray  and  Moses 

[76] :   "What  is  missing  is  a  unified  system  that  integrates 

manpower  planning,  selection,  training,  and  development." 

Amplifying  this,  they  continue: 

There  is  a  heightened  awareness  that  selection  proce- 
dures do  not  operate  in  a  vacuum  but  are  part  of  a  system. 
The  appropriateness  of  a  particular  selection  standard 
depends  not  only  on  the  way  the  job  is  structured,  but 
upon  the  type  and  duration  of  training,  as  well  as  other 
factors.   A  systems  view  of  selection  appears  to  be 
increasing  in  prominence.   Perhaps  we  may  one    day  see  a 
cohesive  theory  of  man  and  woman  at  work. 

.  .  .  Most  of  all,  there  is  a  strong  probability  that 
we  may  move  from  selection  systems  per  se  to  placement 
systems.   To  do  so  will  require  serious  integration  of 
existing  systems,  an  area  which  is  still  in  an  embryonic 
stage . 

Miller  [50]  states  the  need  in  slightly  different 

terms : 

The  development  of  behavioral ly- relevant  job  criteria, 
the  comparison  between  relevant  job  behavior  and  an 
individual's  previous  work  experiences,  and  the  emphasis 
on  planning  an  individual's  work  assignments  in  order  to 
acquire  a  variety  of  samples  of  his  job  behavior  are  all 
integral  elements  of  a  career  mobility  program  and  a 
behaviorally-oriented  identification  program.   There 
should,  therefore,  be  an  interdependence  between  them. 
The  combination  of  an  identification  program  and  a  career 
mobility  pattern  program  should  lead  to  more  accurate 
selection  decisions  and  consequently  more  effective  utili- 
zation of  managerial  personnel. 

Thompson  and  Dalton  [03]  address  performance  appraisal,  and 

suggest  that  a  single  system  is  not  adequate: 

.  .  .  Resist  the  temptation  to  devise  one  grand  per- 
formance appraisal  system  to  serve  all  management  needs. 
Managers  do  have  to  make  decisions  about  assignments, 
promotions,  raises,  and  layoffs,  and  they  also  have  to 
discuss  performance  with  each  man.   There  are  different 
and  sometimes  conflicting  objectives  in  each  of  these 
activities;   tying  them  into  a  single  rationalized  system 
may  make  the  system  less  than  useful  for  any  one  purpose. 
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Management's  own  needs  for  uniformity  and  consistency 
should  not  be  allowed  to  impose  a  rigid  system  that  makes 
impractical  demands  on  the  dynamic  human  organization.   This 
does  not  mean  that  each  of  these  activities  should  be 
planned  in  isolation.   On  the  contrary,  the  entire  method 
of  handling  performance,  salary  increases,  promotions, 
and  job  assignments  must  be  considered  together. 

In  explaining  their  reasons  for  such  a  philosophy,  they  write: 

The  appraisal  system  is  a  very  poor  counseling  tool. 
It's  possible  to  evaluate  a  man's  performance,  but  it's 
very  difficult  to  write  your  evaluation  down  on  paper  and 
then  defend  it  to  the  engineer.    Therefore,  in  the  counseling 
session,  you  try  to  anticipate  how  he'll  respond  to  his 
rating  three  months  later.   If  you  think  lie  will  be  dis- 
satisfied and  complain  about  it,  then  you  only  write  down 
his  faults.   You  don't  tell  him  he's  done  a  good  job  even 
if  he  has.   In  fact  you  don't  counsel  the  man  at  all;  you 
just  try  to  build  a  case  for  the  rating  you  will  later 
be  giving  him. 

The  appraisal  system  favored  by  Thompson  and  Dalton  is  MBO. 

They  compare  it  with  conventional  techniques  as  follows: 

Writers  have  advocated  a  management -by -obj ectives 
appraisal  system  for  many  reasons.  Douglas  McGregor,  for 
example,  favored  it  as  a  way  to  help  a  manager  stop  "playing 
God" --that  is,  judging  the  personal  worth  of  his  fellow 
man.   Alva  F.  Kindall  and  James  Gatza  were  concerned  about 
"quackery" --noting  that  under  conventional  performance 
appraisal,  the  manager  was  asked  to  diagnose  personality 
traits.   For  them,  concentration  on  target- focused  appraisals 
assured  a  healthy  emphasis  on  the  task.   Herbert  H.  Meyer, 
Emanuel  Kay,  and  John  R.  P.  French  were  able  to  show  that 
criticism  had  a  negative  effect  on  achievement  of  goals, 
and  that  performance  improved  most  when  specific  goals 
had  been  established. 

Slevin  [59]  offers  a  model  depicting  the  relationship 

and  contribution  of  the  assessment  center  within  the  larger 

system: 

The  assessment  center  seems  promising  as  a  new  technique 
for  the  selection  of  future  managers.   However,  the  most 
effective  utilization  of  this  technique  will  combine  not- 
only  the  appraisal  function,  but  also  the  management  develop- 
mental function  of  the  center.     The  flow  chart  shown  in 
Figure  (12)  indicates  the  way  in  which  the  assessment  center 
can  be  used  in  a  two  pronged  attack  on  organizational 
development . 
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Figure  12. 

Use  of  the  Assessment  Center  as  an  Appraisal 
and  Development  Tool 
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In  conclusion,  evidence  that  these  concepts  are  not 

merely  theories  but  can  be  attained  in  practice  is  offered 

by  Campbell,  et.  al .  [78]: 

Practices  in  this  company  represent  one  of  the  few 
examples  we  found  of  a  careful  blending  of  validated 
selection  methods  and  the  use  of  informed  and  systematic 
appraisal  methods  for  administering  incentives  and  tailor 
maldng  development  programs.   Thus,  the  testing,  compensa- 
tion, and  salary  systems  as  well  as  the  effective  use  of 
both  company  and  noncompany  training  programs  arc  geared 
toward  identifying,  developing,  and  motivating  effective 
people  to  become  effective  managers. 
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No  other  company  we  visited  had  a  program  the  same 
depth  of  coverage  even  in  single  areas  that  this  company's 
practices  show  in  all  the  key  areas. 

7 .   Unresolved  Problems 

The  report  of  a  seminar  conducted  by  the  Foundation 

for  Research  on  Human  Behavior  [112]  contains  a  listing  of 

questions  for  further  research.   Although  this  list  was 

compiled  in  1957,  this  review  of  the  literature  has  not 

disclosed  significant  progress  on  them  and  they  appear  to 

be  as  timely  as  they  were  in  1957.   The  listing  is  as  follows: 

To  what  extent  is  improvement  possible  in  the  various 
managerial  characteristics  that  are  related  to  job-effec- 
tiveness?  To  what  degree  are  managers  born  and  not  made? 
Are  certain  key  attributes  resistant  to  change?   If  so, 
perhaps  more  emphasis  should  be  placed  on  selection  and 
placement  procedures.   Are  most  key  attributes  capable 
of  being  developed?   If  so,  perhaps  more  emphasis  should 
be  placed  on  training. 

The  future  demand  for  managerial  talent  will  not  permit 
organizations  the  luxury  of  accepting  only  men  with  "no 
weaknesses."   In  what  ways  can  an  executive  be  helped  to 
overcome  his  weaknesses?   What  strengths  can  be  developed 
to  compensate  for  these  weaknesses?   What  characteristics 
in  supervisors,  peers,  and  subordinates  can  be  sought  to 
offset  these  weaknesses?   What  modifications  can  be  made 
in  a  man's  job-description  or  in  organizational  structure 
to  take  these  weaknesses  into  account  and  yet  maximize 
returns  from  his  strengths? 

What  procedures  can  be  used  in  dealing  with  the  "non- 
promotable"  man,  the  man  who  is  adequate  in  his  present 
position  but  has  reached  his  limit?   In  what  ways  can  he 
be  helped  to  lower  his  aspirations  without  destroying  his 
motivation? 

What  procedures  can  be  used  in  dealing  with  the  sub- 
ordinates of  the  "non-promotable"  man?   What  opportunities 
can  be  provided  for  the  subordinates  to  continue  their 
development?   Should  they  be  rotated  through  other  units? 
Should  the  organization  assign  the  non-promotable  super- 
visor activities  which  will  take  him  away  from  his  position 
frequently  and  permit  subordinates  an  opportunity  to  under- 
study him?   How  may  the  tendency  for  low-grade  supervisors 
to  attract  low-grade  subordinates  be  dealt   with?   Arc  there 
ways  in  which  the  non-promotable  man  may  be  made  more  secure 
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in  his  position  so  that  he  will  be  less  threatened  by 
subordinates  and  hence  be  better  able  to  develop  them? 

What  changes  occur  in  an  executive  after  he  assumes  a 
new  managerial  position?   What  effects  do  experience  on  the 
job  and  being  "in  the  role"  have  on  his  competence?   Can 
these  post-promotion  influences  be  predicted? 

In  studying  successful  performance,  how  much  error  is 
introduced  by  contrasting  "more  successful"  and  "less 
successful"  groups  of  managers?   Presumably,  some  random 
situational  factors  (non-promotable  boss)  were  responsible 
for  the  differences  in  achievement  of  the  two  groups.   More 
important,  however,  possibly  some  systematic  factors 
(power  tactics?)  unrelated  to  the  goals  of  the  organization 
were  responsible  for  some  managers  being  promoted  and 
others  not.   How  does  one  take  into  account  the  effects 
of  differential  mobility  in  different  departments?   Does 
test  research  based  on  such  "success"  criteria  tend  to 
perpetuate  errors? 

Are  established  principles  of  management  development 
fully  utilized?   For  example,  are  fledgling  managers  per- 
mitted a  sufficient  proportion  of  "success  experiences" 
early  in  their  career  to  enable  them  to  acquire  self- 
confidence  and  maintain  a  high  level  of  aspiration?   Does 
management  pay  enough   heed  to  the  fundamental  lav:  of 
learning:   heavy  on  reward,  light  on  punishment?   What 
proportion  of  its  management  talent  does  an  organization 
waste  by  using  the  sink-or-swim  method  of  training  managers? 

To  what  degree  does  success  breed  success,  and  failure 
breed  failure?   Does  success  provide  self-confidence  which 
enables  a  man  to  be  more  imaginative,  independent,  original, 
and  assertive?   Does  failure  lead  to  increased  fear  of 
failure  which  prevents  a  man  from  making  decisions,  or 
admitting  ignorance  by  asking  questions?   To  what  degree 
is  success  or  failure  a  circular,  self -perpetuating  process? 
If  so,  what  modifications  in  management  development,  job 
assignment,  etc.,  can  break  the  failure  circle  and  rein- 
force the  success  circle? 

To  what  extent  do  managerial  jobs  impose  rigid  patterns 
of  behavior  even  when  other  equally  satisfactory  ways  of 
achieving  organizational  objectives  exist?   One  participant 
phrased  the  question  as  follows:   Is  managing  like  fighting 
bulls  or  playing  baseball?   The  bullfighter  cannot  achieve 
success  by  merely  killing  the  bull.   He  must  pattern  the 
smallest  details  of  his  performance  to  the  rigidly  held 
expectations  of  a  number  of  people.   The  basebalJ  player, 
on  the  "other  hand,  can  achieve  success  by  maintaining  a 
high  batting  average  or  by  being  an  excellent,  ball  hawk. 
It  doesn't  matter  whether  lie  bats  with  his  "foot  in  the 
bucket,"  or  whether  his  fielding  is  awkward.   All  that 
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counts  are  results.   Is  an  executive  judged  successful  on 
the  basis  of  results  alone  or  must  he  achieve  results  in 
a  certain  way? 

In  summarizing  her  recent  study  of  current  assessment 

programs,  Howard  [34]  identified  future  research  needs  as 

follows : 

A  number  of  issues  have  been  raised  in  this  review  that 
need  to  be  researched,  and  undoubtedly  there  are  many  more. 
A  brainstorming  session  at  the  Center  for  Creative  Leader- 
ship of  the  Smith  Richardson  Foundation  produced  a  list  of 
over  100  relatively  unresearched  questions  concerning  assess 
ment  centers  [38] .   All  the  components  of  the  process  and 
their  integration  can  be  studied  more  intensively,  as  can 
the  potential  uses  of  assessment  procedures  with  such 
organizational  problems  as  personnel  selection,  training, 
motivation,  job  analysis,  job  satisfaction,  and  organi- 
zational climate. 

Aside  from  the  as  yet.  unexplored  research  questions, 
the  importance  of  situation  related  validation  research 
for  those  attempting  to  use  assessment  centers  should  be 
emphasized.   Because  their  initial  industrial  development 
was  grounded  in  research,  it  is  often  casually  implied 
that  therefore  any  .  assessment  center  will  work.   It  is 
true  that  AT5T  laid  a  solid  basic  foundation  in  its 
pioneering  predictive  validity  research.   Some  other  large 
companies  have  also  contributed  to  the  .  .  .  (current 
knowledge  of  assessment  center  validity) . 

Howard  goes  on  to  add  that  future  program  designers  cannot 

expect  to  crowd  under  AT&T's  validity  "umbrella,"  but  rather 

must  tailor  both  program  and  validation  design  to  their 

specific  situation.   She  further  emphasizes  the  importance 

of  utilizing  the  feedback  from  validation   to  refine  the 

assessment  program  and  keep  it  aligned  with  the  demands  and 

dictates  of  the  environment. 

8 .   A  Critique  of  the  Literature 

J.n  the  opinion  of  this  writer,  the  greatest  weakness 
of  the  literature  on  assessment  centers  is  that  it  rests  on 
too  narrow  a  data  base,  e.g.,  although  there  are  a  great 
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number  of  articles,  the  bulk  of  them  are  by  a  relatively  few 
authors  and  about  a  small  number  of  programs.   It  is  speculated 
that  AT$T,  Sohio,  IBM,  AMA,  Sears,  J.  C.  Penney,  and  General 
Electric' s  programs  together  would  account  for  an  overwhelming 
majority  of  the  literature- -indeed,  the  first  three  would 
probably  contribute  40  to  50%  of  the  total. 

Although  the  reason  for  this  is  the  fact  that  the 
concept  is  still  relatively  new,  the  increase  in  numbers  of 
programs  will  not,  by  itself,  soh'e  the  problem,  for  quality 
is  needed  as  well  as  quantity.   This  is  to  say  that  more 
research  of  the  caliber  of  the  Management  Progress  Study  is 
required  if  we  are  to  resolve  the  important  problems  still 
facing  us . 

C.   DESIGN  OF  A  PROTOTYPE  ASSESSMENT  CENTER  FOR  THE  NAVA 
POSTGRADUATE  SCHOOL 

1 .   Assumptions 

a.  Candidates  would  be  U.  S.  Naval  officer  students 
in  their  first  quarter  of  the  817  Management  curriculum  of 
the  Naval  Postgraduate  School. 

b.  Assessors  would  consist  of  a  mixture  of  U.  S. 
Naval  officer  students  and  faculty  of  the  Naval  Postgraduate 
School.   The  student  assessors  would  be  selected  from  students 
in  their  fifth  quarter. 

c.  That  sufficient  flexibility  could  be  obtained 
in  the  curriculum  to  allow  tailoring  of  academic  programs 
to  individuals  in  accordance  with  assessment  findings. 
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d.  That  the  primary  objective  of  the  assessment 
program  would  be  research  (regarding  selection,  placement 
and  development)  with  a  secondary  objective  of  development. 

e.  That  the  program  would  focus  on  the  managerial 
facet  of  the  Operational,  Technical,  Managerial  Systems  (OTMS) 

f .  That  research  data  collected  would  be  privileged 
and  not  divulged  outside  of  the  prototype  assessment  center 
staff. 

2 .   Philosophy 

a.   In  perspective,  assessment  centers  would  only  be 
subsystems  within  the  larger  system  of  personnel  management 
within  the  U.  S.  Navy.   The  assessment  centers  would,  of 
necessity,  interface  with  many  other  subsystems  such  as 
initial  selection,  development,  placement,  selection  for 
promotion,  and  performance  appraisal.   Of  these  (hopefully) 
complementary  subsystems,  performance  appraisal- -like  assess- 
ment involves  the  dual  roles  of  selection  and  development. 

This  writer  proposes  that  appraisal  for  selection 
purposes  should  be  completely  separated  from  appraisal  used 
for  feedback  and/or  development.   Research  has  shown  that 
selection  ratings  are  enhanced  by  confidentiality.   Peer  and 
subordinate  ratings  also  have  potential  in  this  area  and  are 
worthy  of  investigation.   Several  articles  deal  with  this 
subject,  including  three  cited  earlier  in  Section  III  B  h, 
Peer  Ratings  (e.g.,  Kraut  [39],  Amir  et.  al .  [2]  and  Hollander 
[33]).   All  of  these  supported  the  use  of  peer  ratings  for 
prediction  of  later  performance.   Githens  and  bister  [113] 
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found  subordinate  ratings  to  have  potential,  but  that  their 
use  was  considered  threatening  to  the  majority  of  Naval  officers 
surveyed.   Ideally,  selection  appraisals  should  be  specifically 
designed  to  complement  the  assessment  center  results  and 
should  concentrate  on  those  dimensions  which  can  be  best 
observed  on  the  job. 

This  writer  offers  management  by  objectives  (MBO) 
as  the  best  vehicle  for  the  developmental  aspect  of  performance 
appraisal.   Articles  on  MBO  abound  in  the  literature,  however 
McGregor's  classic  [47]  remains  one  of  the  best.   He  condemned 
"conventional"  performance  appraisal  and  recommended  analysis 
vs.  appraisal  where  the  superior  would  act  as  a  coach  instead 
of  a  judge. 

b.   In  the  "ideal"  system,  as  envisioned  by  this 
writer,  more  than  one  assessment  would  be  performed  during 
a  career  and  the  objectives  of  each  would  differ.   The  concept 
could  be  modeled  as  shown  in  Figure  13.   The  assessments 
indicated  in  the  model  are  explained  as  follows: 

(1)  Career  Entry.   Since  no  performance  appraisal 
information  is  available  (or  at  least  none  in  the  job  being 
applied  for)  the  hiring  decision  would  be  based  solely  on 
assessment  data.   This  process  would  involve  large  numbers 
of  candidates,  thus,  would  have  to  be  as  short  as  possible 
while  still  obtaining  valid  results.   The  ATfiT  one-day  Early 
Identification  Program  described  by  Moses  [51]  is  an  example 
of  this  type  of  program.   Selection  would  be  the  primary 
objective,  and  the  brevity  of  the  program  would  virtually 
rule  out  any  developmental  aspects. 
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Figure  13. 
Number  and  Nature  of  Assessments  During  Career 
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(2)   Career  Commitment.   This  point  would  occur 
after  four  years  of  service,  when  the  officer  had  completed 
his  obligated  service  and  elected  to  remain  in  the  service 
as  a  career.   Such  a  candidate  would  be  in  the  rank  of 
Lieutenant  and,  if  a  line  officer,  would  have  completed  his 
first  operational  (sea  duty)  tour  and  be  commencing  his  first 
tour  of  shore  duty.   He  would  soon  be  faced  with  the  decision 
as  to  which  career  path  to  choose  (Operational,  Technical, 
or  Managerial),  and  an  assessment  should  provide  information 
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not  obtained  in  his  performance  appraisals  which  would  enhance 
the  quality  of  this  decision- -to  the  benefit  of  the  individual 
and  the  U.  S.  Navy. 

(3)  Selection  for  Initial  Command.   By  this  stage 
of  a  career  the  difference  in  career  paths  make  any  "norm" 
selected  a  little  less  representative,  however  it  should  occur 
at  about  12  -  15  years  of  service  at  a  rank  of  senior  Lieu- 
tenant Commander  or  junior  Commander.   This  decision  is  a 
crucial  one  for  the  individual  as  well  as  the  organization. 
Since  it  represents  a  marked  change  in  the  degree  of  responsi- 
bility and  scope  of  decisions  required  of  the  officer,  an 
assessment  can  help  combat  the  "Peter  Principle"  by  probing 
those  key  factors  of  command  which  have  not  yet  been  evaluated 
on  the  job. 

( 4 )  Selection  for  Major  Command/Flag  Rank  or 
Early  Retirement.   Once  again  the  degree  of 

responsibility  and  scope  of  decisions  take  a  quantum  jump, 
therefore  the  benefits  of  assessment  cited  in  the  previous 
section  would  apply   here  as  well.   For  officers  retiring 
early,  an  assessment  could  provide  them  with  an  inventory 
of  their  attributes  to  assist  them  in  deciding  on  a  second 
career. 

(5)  Retirement .   Same  considerations  as  early 
retirement  previous])'  described.   This  could  be  viewed  as  a 
sort  of  "Project  Transition,"  and  although  the  direct  benefits 
would  accrue  to  the  individual,  the  organization  might  benefit 
indirectly  from  such  a  "personalized"  personnel  management 
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policy,  e.g.,  such  a  benefit  might  make  a  Naval  career  much 
more  attractive  to  young  officers  thereby  enhancing  the  overall 
quality  of  the  officer  corps. 

It  should  be  noted  that  the  emphasis  is  on  selec- 
tion initially,  but  there  is  a  gradual  shift  toward  development 
as  the  career  progresses,  until  at  the  end,  inventory  and/or 
development  are  the  primary  objectives. 

Another  potential  objective  is  that  of  placement. 
By  ascertaining  an  officer's  strengths  and  weaknesses  with  an 
acceptable  degree  of  accuracy,  it  would  be  possible  to  place 
him  in  billets  better  suited  to  his  talents --as  advocated  by 
Fiedler  f 19] - -thereby  making  better  use  of  the  human  resources 
of  the  Navy. 

c.  Assessment  programs  can.  be  classified  according 

to  their  use  of  signs  and/or  samples.   "Psychometric"  programs 

emphasize  signs,  while  at  the  other  end  of  the  continuum, 

1 
"behavioral"  programs  emphasize  samples.    Most  current  programs 

fall  nearer  the  middle  of  the  spectrum  and  assess  both  signs 

and  samples. 

d.  Design  of  the  assessment  program  should  be  based 
on  the  best  available  information.   In  this  writer's  opinion, 
the  best  base  would  be  data  obtained  from  a  quality  research 
effort.   An  alternative  would  be  to  obtain  the  data  from  a 
panel  of  "experts,"  however  the  validity  of  such  data  would 
be  questionable. 


The  author  realizes  that  ail  signs  are  samples,  but 
whether  or  not  a  sample  is  a  s.ign  is  a  matter  determined  via 
validation  efforts. 
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A  decision  would  have  to  be  made  as  to  whether 
an  officer  could  be  selected  on  specific  dimensions  unique 
to  a  billet  (or  family  of  billets)  or  whether  the  heterogeneity 
and  number  of  billets  would  force  an  assessment  of  the  "whole 
personality"  on  general  dimensions  as  was  done  by  the  O.S.S. 
[99].   This  decision  would  properly  require  thorough  job 
analyses  which  would  define  the  billets  and  enable  judgments 
as  to  which  (if  any)  might  be  combined  into  "job  families." 
These  job  analyses  should  be  behaviorally-oriented  to  identify 
significant  job  dimensions  which  can  be  measured  by  observable 
behaviors  (for  subsequent,  design  of  the  assessment  program)  . 
3 .   Program  Considerations 
a.   Objectives 

(1)  Primary .   The  primary  objective  of  the  pro- 
totype assessment  center  would  be  research  focusing  on   the 
selection  and  placement  of  managers.   Virtually  all  of  the 
data  obtained  on  this  aspect  would  be  held  in  strict  confi- 
dence by  the  assessment  center  staff. 

(2)  Secondary .   A  secondary  objective  would  be 
the  individual  development  of  participants,  both  candidates, 
and  student  assessors.   The  goal  would  be  to  produce  a  per- 
sonal development  plan  for  each  candidate  which  could  be 
translated  into  an  individual  curriculum.   Although  the  sub- 
stantive differences  between  individual  curricula  might  be 
small,  the  participative  involvement  of  the  student  in  the 
determination  of  Ids  academic  program  could  produce  signifi- 
cant improvement  in  satisfaction,  motivation  and,  utlimately, 
performance . 
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b.  Content 

For  a  research-oriented  program  such  as  this,  a 
mixture  of  signs  and  samples  is  considered  to  be  the  prudent 
choice.   This  would  offer  a  full  range  of  instruments  for 
validation  and  enable  the  validation  process  to  indicate  which 
are  most  effective.   Any  restriction  of  this  (i.e.,  using 
only  signs  or  samples  to  the  exclusion  of  the  other) ,  would 
amount  to  "pre-selection"  decisions  without  scientific  basis. 

c.  Length 

In  view  of  the  academic  schedule  which  exists  at 
the  Naval  Postgraduate  School,  this  writer  considers  the 
five-day  program  to  be  the  best  length.   This  would  roughly 
break  down  as  three  days  of  assessment  followed  by  two  days 
of  evaluation  and  report  writing  by  the  assessors. 

d.  Administration 

The  academic  schedule  poses  some  challenging 
scheduling  problems.   One  solution  would  be  to  make  the 
fifth  quarter  of  the  management  curriculum  a  "vertical" 
schedule  for  those  students  selected  as  assessors  (e.g.,  the 
courses  would  be  taken  consecutively  vice  simultaneously). 
Assessor  training  could  be  conducted  during  the  first  week 
of  the  quarter,  and  then  assessment  of  twelve  candidates  per 
week  conducted  until  all  candidates  had  been  assessed.   Since 
the  candidates  would  only  be  involved  for  three  days,  their 
other  course  work  could  be  scheduled  lightly  until  assessments 
were  completed  and  then  the  tempo  increased  to  compensate. 
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D.   DESIGN  OF  VALIDATION/EVALUATION/FEEDBACK  SYSTEM 

These  systems  would  differ  according  to  the  objectives 

of  the  program  to  be  evaluated.   Their  design  should  be 

accomplished  at  the  outset  before  the  assessment  program 

design  is  undertaken  as  recommended  in  the  O.S.S.  "Assessment 

of  Men"  [99]  as  follows: 

.  .  .  appraisal  is  the  target  of  assessment,  and  it  is 
alwa)  5  well  to  see  your  target  before  taking  aim.   One 
should  know  ahead  of  time,  for  example,  whether  the  target 
will  be  an  objectively  determined  figure,  such  as  the 
number  of  units  of  work  a  man  completes  per  day,  or  a 
rating  of  his  efficiency  given  by  competent  observers  .  .  . 

The  point  that  is  being  stressed  here  is  that  whatever 
the  determinants  of  appraisal  may  be,  it  is  important  to 
distinguish  them  at  the  start,  so  that  the  assessment 
program  can  be  designed  to  take  account  of  them. 

It  must  be  pointed  out  that  the  foregoing  tacitly  assumes 

the  elements  of  appraisal  selected  to  be  relevant  Lo  the 

objective(s)  of  the  assessment. 

Once  developed,  the  appraisal  system  should  be  tested 
on  a  number  of  "successful"  members  of  the  organization, 
if  possible,  and  then  revised  as  necessary. 

A  final  consideration  is  that  the  rating  scales  for  the 
assessment  and  appraisal  programs  should  be  identical  on 
those  dimensions  which  are  shared  by  both. 


I V .   JUSTIFICATION 

A.  OF  ASSESSMENT  CENTERS 

The  Management  Progress  Study,  and  other  ATfjT  studies 
which  controlled  for  contamination,  offer  the  most  rigorous 
proof  of  the  concept.   In  addition  to  these,  one  must  acknow- 
ledge the  amount  of  resources  devoted  to  assessment  centers 
by  large  profit-making  organizations.   This  is  not  to  advocate 
a  blind  policy  of  follow- the -leader ,  but  the  size  of  the  com- 
mitments made  (5  million  dollar  annual  budget  by  AT§T  [53]) 
are  worthy  of  serious  consideration,  since  no  organization 
would  commit  such  resources  to  a  program  unless  they  felt 
they  were  getting  a  reasonable  return.   AT§T  has  supported 
the  concept  for  seventeen  years  and  continued  to  increase  its 
commitment  with  each  passing  year.   The  British  and  German 
military  programs  also  represent  heavy  commitments  of  long 
standing. 

B.  FOR  ESTABLISHMENT  OF  A  PROTOTYPE  ASSESSMENT  CENTER  IN 
THE  U.  S.  NAVY 

If  one  accepts  the  concept  as  valid,  then  the  remaining 
question  becomes:  How  to  best  utilize  it  within  the  Navy-- 
vice  whether  or  not  it  should  bo. 

Some  interesting  and  prophetic  remarks  in  this  regard 
were  made  by  Rear  Admiral  I.  E.  llobbs  to  students  of  the 
U.  S.  Navy  Management  School  at  Monterey  [315]  to  wit: 
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Perhaps  we  should  take  a  good  look  at  what  the  Army  is 
doing  experimentally  in  evaluation  procedures  to  try  and 
measure  an  officer's  interest  and  abilities  in  three  fields 
of  combatant,  administrative,  and  technical.   This  is 
being  done  by  two  full  days  of  achievement  testing  and 
then  validated  by  actual  field  performance  tests  two  to 
five  years  later. 

This  method  of  testing  is  somewhat  similar  to  the 
German  method  used  after  World  War  I  and  used  by  the  Bri- 
tish and  O.S.S.  during  World  War  II.   It  has  considerable 
possibilities  and  should  be  carefully  monitored  by  the 
Navy  as  a  means  of  identifying  truly  outstanding  officers 
in  the  fields  of  operations,  planning  and  technical  branches 

•    •    • 

The  Air  Force,  through  a  special  contract  with  the 
University  of  California,  is  conducting  some  very  signif- 
icant tests  in  evaluation  of  officers  at  their  Squadron 
Officer  School  (AF  opposite  to  the  Navy  General  Line 
School)  located  at  Maxwell  Air  Force  Base.   This  effort 
consists  of  student  peer  ratings,  situational  testing 
during  escape  and  evasion,  instructor  evaluation  and 
other  psychological  procedures  and  techniques  which  are 
available  but  not  generally  employed.   Through  monitoring 
and  observing  these  efforts  the  Navy  might  well  learn 
additional  techniques  of  identifying  naval  officers  to 
man  our  fleet  in  the  post  1970  era. 

The  prophesy  was  the  mention  of  the  fields  of  "operations, 
planning  and  technical  branches"  which  recently  became  reality 
as  the  Operational,  Technical,  Managerial  System  (OTMS) .   The 
irony  is  that  these  remarks  - -timely  as  they  sound  today- -were 
made  on  20  February,  1959.   In  the  ensuing  years,  assessment 
centers  have  not  been  disproved  or  lost  favor,  but  rather,  as 
Bray  [  68  ]  puts  it:   "...  have  reached  the  critical -mass 
state  and  are  ready  to  take  off  as  more  companies  use  them." 
This  writer  submits  that  the  advent  of  OTMS  and  the  All- 
Volunteer  Force  have  made  personnel  decisions  even  more 
important  than  they  were  in  the  past  and  increase  the  potential 
value  of  the  assessment  center  to  the  U.  S.  Navy. 
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C.   FOR  ESTABLISHMENT  OF  A  NAVY  PROTOTYPE  ASSESSMENT  CENTER 
AT  THE  NAVAL  POSTGRADUATE  SCHOOL  (NPS) 

The  Naval  Postgraduate  School  offers  a  unique  combination 
of  candidates  and  assessors  from  a  wide  variety  of  specialties, 
backgrounds  and  ranks;  and  necessary  supporting  facilities. 
These  facilities  include  videotape  capabilities  as  well  as  a 
full  range  of  other  educational  media  devices,  and  a  computer 
center.   Some  other  significant  factors  are  the  expertise  of 
the  faculty  (from  Naval  officers  with  operational  experience 
to  psychologists,  psychometricians ,  etc.)  and  the  liaison 
with  NPS  curriculum  sponsors  for  continuous  feedback  on  changes 
in  billet  requirements.   In  support  of  the  objective  of  develop- 
ment, the  school  offers  courses  in  a  wide  variety  from  basic 
skills  such  as  reading  labs,  speech  courses,  writing,  etc.; 
to  more  advanced  work  dealing  with  aspecLs  of  leadership, 
interpersonal  skills,  organizational  development,  problem 
solving,  decision  making  and  human  resource  management. 

In  considering  the  impact  of  a  prototype  assessment  center 

upon  the  NPS,  the  following  three  objectives  of  the  school 

are  considered  to  be  pertinent: 

Foster  an  organization  and  provide  supporting  incentives 
at  NPS  in  order  to  optimize  utilization  of  resources  to 
meet  the  learning  requirements  of  the  students. 

Provide  for  continued  studies  in  new  programs,  new 
techniques  of  teaching,  better  definitions  of  learning 
objectives  and  measures  of  effectiveness  of  our  educational 
programs. 

Provide  for  increased  regard,  for  individual  diversity 
and  needs  on  the  part  of  students  and  provide  an  improved 
working  environment  for  them. 
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This  writer  submits  that  significant  aspects  of  all  three 
of  these  objectives  could  be  met  by  the  proposed  prototype 
assessment  center.   These  benefits  could  go  far  toward  off- 
setting the  costs  of  the  program.   Also,  because  many  of  the 
costs  represent  sunk  costs  (computer  center,  educational 
media  facilities,  faculty,  etc.)  the  actual  out-of-pocket 
costs  would  be  relatively  low  as  compared  to  an  industrial 
organization  which  had  none  of  these. 

In  summary,  the  prototype  assessment  center  could  be 
established  most  efficiently  at  the  Naval  Postgraduate  School 
and  could  benefit  the  school  and  the  U.  S.  Navy  by  its 
existence . 
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V.   TASKS  TO  BE  PERFORMED 

A  review  of  section  III  yields  the  following  tasks  yet 
to  be  performed:   (The  Interviews  and  Literature  Review  are 
considered  complete  as  of  this  writing.) 

1.  Selection  of  criteria  to  be  utilized  in  the  evaluations 
This  would  have  to  be  done  separately  for  each  of  the  assess- 
ment objectives. 

2.  Job  analyses  of  jobs  (billets)  for  which  candidates 
would  be  assessed.   Observable  behaviors  should  be  considered 
separately,  as  they  would  be  of  special  interest  in  the  design 
of  the  simulations. 

3.  Based  on  the  foregoing  steps,  a  decision  must  be  made 
as  to  the  dimensions  to  be  assessed,  and  further,  on  the  number 
of  different  assessment  programs  which  would  be  required.  If 
there  is  a  great  deal  of  heterogeneity  among  the  jobs,  a  large 
number  of  separate  assessment  programs  would  be  called  for. 

A  more  practical  alternative  to  this  would  be  a  sing1   program 

based  on  "general"  dimensions  which  would  attempt  to    .ess 

the  Gestalt  or  whole  personality  as  was  done  by  the  O.S.S.  [99], 

4.  Selection/design  of  instruments  to  measure  the  dimen- 
sions selected  (in  step  3).   In  the  prototype  program  consider- 
able overlap  would  be  expected  since,  whenever  possible,  com- 
peting instruments  would  be  included  to  allow  validation  on 
both  of  them  (e.g.,  provide  selection  on  a  scientific  basis 
vice  hunch  or  conjecture). 
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5.  Development  of  rating  scales  for  use  in  both  the  assess- 
ment and  evaluation  programs.   All  dimensions  common  to  both 
programs  should  have  identical  scales. 

6.  Perform  "trial  runs"  to  test  all  elements  of  the  assess- 
ment program  which  require  testing  (e.g.,  instruments  designed 
for  the  program).   These  runs  should  ideally  employ  "successful" 
members  of  the  organization  (U.  S.  Navy)  as  test  candidates. 

7.  Make  any  revisions  to  the  program  as  dictated  by 
results  of  step  6. 

8.  Develop  an  assessor  training  program. 

9.  Test  the  assessor  training  program. 

10.   Revise  the  assessor  training  program  as  necessary. 
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VI.   ANTICIPATED  RESULTS 

A.  SHORT  TERM  (Less  than  2  years) 

1.  An  evaluation  of  the  development  objective  of  the 
prototype  assessment  center. 

2.  Improve  the  effectiveness  and  efficiency  of  the  NP5 
by  enhancing  the  attainment  of  stated  goals  (as  discussed  in 
Section  IV) . 

3.  Increased  personal  development  of  student  assessors 
and  candidates  who  participate  in  the  program. 

4.  Development  of  an  in-house  expertise  in  the  design 
and  operation  of  an  assessment  center. 

B.  LONG  TERM  (More  than  2  years) 

1.  Evaluation  of  the  selection  and  placement  objectives 
of  the  prototype  assessment  center. 

2.  If  the  assessment  center  proved  to  be  effective,  the 
ultimate  result  could  be  improved  utilization  of  human  resources 
within  the  Navy.   If  one  accepts  the  premise  that  people  are 
the  decisive  factor  in  combat,  the  significance  of  this  result 
is  obvious. 


152 


BIBLIOGRAPHY 


Articles 


[1]  Albrook,  R.  C. ,  How  to  spot  executives  early:   assessment 
centers.   Fortune,  1968,  78,  106-108. 

[2]  Amir,  Y.,  Kovarsky,  Y.  §  Sharan,  S .,  Peer  nominations  as 
a  predictor  of  multistage  promotions  in  a  ramified  organ- 
ization.  Journal  of  Applied  Psychology,  1970,  54(5), 
462-469.  ' 

\%\    Bender,  J.  M. ,  What  is  "typical"  of  assessment  centers? 
Personnel,  1973,  50,  50-57. 

[4]  Berkwitt,  G.  J.,  The  big  new  move  to  measure  managers. 
Dun's  Review,  1971,  62-66. 

[5]  Boehm,  V.  R. s  Negro-white  differences  in  validity  of 
employment  and  training  selection  procedures:   summary 
of  research  evidence.   Journal  of  Applied  Psychology, 
1972,  56(1),  33-39. 

[6]  Bowling,  R.  A.,  Battle -mindedness .   United  States  Naval 
Institute  Proceedings,  1972,  98(10/836),  41-45. 

[7]  Bray,  D.  W.  ,  Comment:   the  management  progress  study. 
American  Psychologist,  1964,  19(6),  419-420. 

[8]  Bray,  D.  W. ,  The  assessment  center:   opportunities  for 
women.   Personnel ,  19  71,  33-34. 

[9]  Bray,  D.  W. ,  and  Campbell,  R.  J.,  Selection  of  salesmen 
by  means  of  an  assessment  center.   Journal  of  Applied 
Psychology,  1968,  52(1),  36-41. 

[lOp-ifray,  D.  W.  and  Grant,  D.  L.,  The  assessment  center  in 
the  measure  of  potential  for  business  management. 
Psychological  Monographs ,  1966,  80(17  whole  no.  625),  1-27 

[11]  Byham,  W.  C,  Assessment  centers  for  spotting  future 
managers.   Harvard  Business  Review,  1970,  150-160. 

[12]  Byham,  W.  C.  and  Pentecost,  R..,  The  assessment  center: 
identifying  tomorrow's  managers.  Personnel ,  1970,  47, 
17-28. 

[13]  Byham,  W.  C.  and  Thornton,  G.  C.  Ill,  Assessment  centers: 
a  new  aid  in  management  selection.   Studies  in  Personnel 
Psychology,  1970,  2(2),  21-35.. 

153 


[14]  Campbell,  R.  J.  and  Bray,  D.  W.,  Assessment  centers:   an 
aid  in  management  selection.   Personnel  Administration , 
1967,  7-13. 

[15]  Carleton,  F.  0.,  Relationships  between  follow-up  evaluations 
and  information  developed  in  a  management  assessment  center. 
Proceedings,  78th  Annual  Convention,  APA,  19  70. 

[16]  Cohen,  B.  M.  and  Jaffee,  C.  L.,  Assessment  centers  aid 

government  agencies.   Government  Executive,  1973,  42-44. 

[17]  Cristian-Melicson,  G.,  Decision-making,  problem  solving 
and  creative  thinking.   Studia  Psvchologica ,  1972,  14, 
59-63. 

[18]  Dodd,  W.  E.,  Will  management  assessment  centers  insure 

selection  of  the  same  old  types?   Proceedings,  78th  Annual 
Convention,  APA,  1970,  569-570. 

[19]  Fiedler,  F.  E.,  Engineer  the  job  to  fit  the  manager.   Harvard 
Business  Review,  1965,  Sep -Oct,  115-122. 

[20]  Finley,  R.  M. ,  Jr.,  Evaluation  of  behavior  predictions 
from  projective  tests  given  in  a  management  assessment 
center.   Proceedings,  78th  Annual  Convention,  APA,  1970. 

[21]  Flanagan,  J.  C,  Some  considerations  in  the  development 
of  situational  tests.   Personnel  Psychology,  1954,  7, 
461-464. 

[22]  Frederiksen,  N.,  Saunders,  D.  R. ,  and  Wand,  B.,  The  in- 
basket  test.   Psychological  Monographs,  1957,  71(9  whole 
No.  438),  1-28. 

[23]  Ghiselli,  E.  E.,  The  validity  of  a  personnel  intervie)\r. 
Personnel  Psychology,  1966,  19,  389-394. 

[24]  Ghiselli,  E.  E.  and  Haire,  M. ,  The  validation  of  selection 
tests  in  the  light  of  the  dynamic  character  of  criteria. 
Personnel  Psychology,  1960,  13(3),  225-231. 

[25]  Ginsburg,  L.  R.  and  Silverman,  A.,  The  leaders  of  tomorrow: 
their  identification  and  development.   Personnel  Journal, 
1972,  51,  662-666. 

[26]  Grant,  D.  L.  and  Bray,  D.  W. ,  Contributions  of  the  interview 
to  assessment  of  management  potential.   Journal  of  Applied 
Psychology,  1969,  53(1),  24-34. 

[27]  Grant,  D.  L.,  Katkovsky,  W.,  and  Bray,  D.  W.,  Contributions 
of  projective  techniques  to  assessment  of  management 

potential.   Journal of  Applied  Psychology,  1967,  51(3), 

226-232. 


154 


[28]  Greenwood,  J.  M.  and  McNamara,  W.  J.,  Interrater  reliability 
in  situational  tests.   Journal  of  Applied  Psychology,  1967, 
51,  101-106. 

[29]  Guion,  R.  M.  and  Cottier,  R.  F.,  Validity  of  personality 
measures  in  personnel  selection.   Personnel  Psychology, 
1965,  18,  135-164. 

[30]  Hall,  D.  T.,  A  theoretical  model  of  career  subidentity 
development  in  organizational  settings.   Organizational 
Behavior  and  Human  Performance,  1971,  6,  50-76. 

[31]  Hardesty,  D.  L.  and  Jones,  W.  S.,  Characteristics  of 

judged  high  potential  management  personnel - -the  operations 
of  an  industrial  assessment  center.  Personnel  Psychology, 
1968,  21,  85-98, 

\^\    Hinrichs,  J.  R.,  Comparison  of  "real  life"  assessments  of 
management  potential  with  situational  exercises,  paper- 
and-pencil  ability  tests,  and  personality  inventories. 
Journal  of  Applied  Psychology,  1969,  53(5),  425-432. 

[33]  Hollander,  E.  P.,  Validity  of  peer  nominations  in  pre- 
dicting a  distant  performance  criterion.   Journal  of 
Applied  Psychology,  1965,  49(6),  434-438. 

[^4-^Howard,  A.,  An  assessment  of  assessment  centers.   Academy 
of  Management  Journal,  1974,  17(1),  115-13  4. 

[35]  Jaffee,  C.  L.,  Assessment  centers  help  find  management 

potential.   Bell  Telephone  Magazine,  1965,  44(3),  18-25. 

[36]  Jaffee,  C.  L.,  Cohen,  S.  L.  and  Cherry,  R. ,  Supervisory 

selection  program  for  disadvantaged  or  minority  employees. 
Training  and  Development  Journal ,  1972,  26(1),  22-27. 

[37]  Jelks,  E.  B.,  The  assessment  center.   Administrative 
Management,  1971,  67-6  8. 

[38]  Jeswald,  T.  A.,  Research  needs  in  assessment- -a  brief 

report  of  a  conference.   The  Industrial  Psychologist,  1971, 
9(1),  12-14. 

[39]  Kraut,  A.  I.,  Use  of  managers'  peer  ratings  to  predict 
executive  success.   Proceedings  of  the  78th  Annual 
Convention,  APA ,  19  71. 

[40]  Kraut,  A.  I.,  A  hard  look  at  management  assessment  centers 
and  their  future.   Personnel  Journal,  1972,  317-326. 

[41^Kraut,  A.  I.  and  Scott,  G.  J.,  Validity  of  an  operational 

management  assessment  program.   Journal  of  Applied  Psychol_- 
ogy,  1972,  56(2),  124-129. 


155 


[42]  Maier,  N.  R.  F.,  Effects  of  training  on  decision-making. 
Pscyhological  Reports,  1972,  30,  159-164. 

[43]  McConnell,  J.  H.,  The  assessment  center  in  the  smaller 
company.   Personnel ,  1969,  40-46. 

[44]  McConnell,  J.  H. ,  The  assessment  center:   a  flexible 
program  for  supervisors.  Personnel ,  1971,  34-40. 

[45^McConnell ,  J.  H.,  Sizing  up  assessment  centres.   Personnel 
Management,  1973,  5,  36-38. 

[46]  McConnell,  J.  H.  and  Parker,  T.  C,  An  assessment  center 

program  for  multi-organizational  use.   Training  and  Develop 
inent  Journal,  1972,  26(3),  6*14, 

[47]  McGregor,  1).,  An  uneasy  look  at  performance  appraisal. 
Harvard  Business  Review,  1972,  133-138. 

[48]  Meyer,  H.  II.,  The  validity  of  the  in-basket  test  as  a 

measure  of  managerial  performance.   Personnel  Psychology, 
1970,  23,  297-307. 

[49]  Meyer,  H.  H.,  Kay,  E.  and  French,  J.  R.  P.  Jr.,  Split  roles 
in  performance  appraisal.   Harvard  Business  Review,  1965, 
Jan-Feb,  123-129. 

[50]  Miller,  E.  L. ,  Identifying  high-potential  management  per- 
sonnel.  Management  of  Personnel  Quarterly,  19  69,  8(4), 
14-18. 


[  5  ]<[  Moses,  J.  L.,  The  development  of  an  assessment  center  for 
the  early  identification  of  supervisory  potential. 
Personnel  Psychology,  1973,  26,  56  9-580. 

[52]  Parry,  S.  B.,  The  name  of  the  game  is  simulation.   Training 
and  Development  Journal,  1971,  25(2),  28-32. 

[53]  Pomerleau,  R. ,  Identification  systems:   the  key  to  effec- 
tive manpower  planning.   Personnel  Journal,  1973,  434-4  41. 

[54]  Prien,  E.  P.,  Assessments  of  higher-level  personnel:   Y. 

an  analysis  of  interviewers'  predictions  of  job  performance 
Personnel  Psychology,  1962,  15,  319-334. 

[55]  Sakoda,  J.  M. ,  Factor  analysis  of  OSS  situational  tests. 
Journal  of  Abnormal  and  Social  Psychology ,  1952,  8  4  3-852. 

[56]  Sawyer,  J.,  Measurement  and  prediction,  clinical,  and 

statistical.   Psychological  Bulletin,  1966,  66,  178-200. 

[57]  Schmahling,  E.,  The  staff  officer  and  selection  course 

of  the  Navy  closed  its  doors.   Truppenpraxis ,  April  19  7  4, 
303-306. 


156 


[58]  Senger,  J.  D.,  Managers'  perception  of  subordinates' 

competence  as  a  function  of  personal  value  orientations. 
Academy  of  Management  Journal,  19  71,  14(4),  415-423. 

[59]  Slevin,  D.  P.,  The  assessment  center:   breakthrough  in 

management  appraisal  and  development.   Personnel  Journal, 
1972,  255-261. 

[60]  Streufert,  S.,  Clardy,  M.  A.,  Driver,  M.  J.,  Karlins,  M., 
Schroder,  H.  M.  ,  and  Suedfeld,  P.,  A  tactical  game  for 
the  analysis  of  complex  decision  making  in  individuals 
and  groups.   Psychological  Reports,  1965,  17,  723-729. 

[61]  Taft,  R. ,  Multiple  methods  of  personality  assessment. 
Psychological  Bulletin,  1959,  56(5),  333-352. 

[62]  Thompson,  D.  B.,  Pick  them  young--then  push.   Industry 
Week,  197  2,  37-41. 

[63]  Thompson,  P.  H.  and  Dalton,  G.  W.,  Performance  appraisal: 
managers  beware.   Harvard  Business  Review,  19  70,  14  9-157. 

[64]  Thomson,  H.  A.,  Comparison  of  predictor  and  criterion 

judgments  of  managerial  performance  using  the  multitrait- 
multimethod  approach.   Journal  of  Applied  Psychology,  1970, 
54,  496-502. 

[65]  Veaudry,  W,  F.  and  Campbell,  J.  C,  A  new  look  at  leader- 
ship evaluation.   Army,  Feb.  1974,  23-26. 

[66]  Weber,  R.  J.,  Effects  of  videotape  feedback  on  task  group 
behavior.   Proceedings,  79th  Annual  Convention,  APA,  1971, 
499-5001 

[67]  Werniinont,  P.  F.  and  Campbell,  J.  P.,  Signs,  samples  and 
criteria.   Journal  of  Applied  Psychology,  1968,  52(5), 
372-576. 

[68]  Where  they  make  believe  they're  the  boss.  Business  Week, 
August  28,  1971,  34-35. 

[69]  Wikstrom,  W.  F.,  Assessing  managerial  talent.   The  Con- 
ference Board  Record,  National  Industrial  Conference 
Board,  March,  196  7. 

[70]  Wilson,  J.  E.  and  Tatge,  IV.  A.,  Assessment  centers  - -further 
assessment  needed?   Personnel  Journal,  1973,  52,  172-179. 

[71]  Wollowick,  H.  B.  and  McXamara,  W.  J.,  Relationship  of  the 
components  of  an  assessment  center  to  management  success. 
Journal  of  Applied  Psychology,  196  9,  53(5),  548-352. 

\rf\    Worbois,  G.  M. ,  Predicting  performance  of  substation 

operators.   Journal  of  Applied  Psychology,  1951,  35,  15-19. 


157 


Books 


[73]  Albright,  L.  E.,  Glennon,  J.  R.  and  Smith,  \\r .  J.,  The  use 
of  psychological  tests  in  industry.   Cleveland:   Howard 
Allen,  Inc.,  1963. 

[74]  Bellows,  R.  M. ,  Gilson,  T.  Q.  and  Odiorne,  G.  S.,  Executive 
skills,  their  dynamics  and  development.  Englewood  Cliffs, 
N.  J.:   Prentice-Hall,  Inc.,  1962. 

[75]  Bray,  D.  W.,  The  assessment  center  method  of  appraising 

management  potential.   In  J.  W.  Blood  (Ed.),  The  personnel 
job .   New  York:   American  Management  Association,  1964. 

[76]  Bray,  D.  W.  and  Moses,  J.  L.,  Personnel  selection.   In 

P.  H.  Mussen  and  M.  R.  Rosenzweig  (Eds.),  Th e  Annual  Re v i ew 
of  Psychology.   Palo  Alto:   Annual  Reviews,  Inc.,  1972. 

[7  7]  Byham,  W.  C,  The  uses  of  personnel  research.   New  York: 
American  Management  Association,  Inc.,  1968. 


8]  Campbell,  J.  P.,  Dunnette,  M.  D.,  Lawler,  E.  E.  and  Weick, 
K.  E.,  Managerial  behavior,  performance,  and  effectiveness. 
New  York :   McG raw"- Hill,  1 9 "70  .  " 

[79]  Compton,  H.  P.  and  Bennett,  W. ,  Communication  in  supervisory 
management .   Camden,  N.  J.:   Thomas  Nelson  and  Sons,  1967. 

[80]  Copeman,  G.,  The  chief  executive  and  business  growth. 
New  York:   Leviathan  House,  1971. 

[81]  Dunnette,  M.  D.,  Personnel  selection  and  placement.   Belmont, 
California:   Wadsworth  Publishing  Company,  Inc.,  1966. 

[82]  Dunnette,  M.  D.,  Multiple  assessment  procedures  in  identi- 
fying and  developing  managerial  talent.   In  P.  McReynolds 
(Ed.),  Advances  in  psychological  assessment,  Vol.  II  .  P  a  1  o 
Alto,  California:   Science  and  Behavior  Books,  1971. 

[83]  Edwards,  A.  L.  and  Abbott,  R.  D.,  Measurement  of  personality 
traits:   theory  and  technique.   In  P.  H.  Mussen  and  M.  R. 
Rosenzweig  (Eds.),  The  Annual  Review  of  Psychology.   Palo 
Alto:   Annual  Reviews,  Inc.,  1973. 

[84]  Parago,  0.  and  Gittler',  L.  F.  (Eds).,  German  psychological 
warfare:   survey  and  bibliography.   New  York:   Comm.  for 
Nat.  Morals,  lUtl . 


I  r,  8 


Finkle,  R.  15.  and  Jones,  W.  S.,  Assessing  corporate  talent: 
a  key  to  managerial  manpower  planning.   New  York:   John 
Wiley  and  Sons,  Inc.,  1970. 

Ghiselli,  E.  E.,  The  validity  of  occupational  aptitude 
tests .   New  York:   John  Wiley  and  Sons,  Inc.,  1966. 

Guion,  R.  M. ,  Personnel  testing.   New  York:   McGraw-Hill, 
1965. 

Huse,  E.  F.  and  Bowditch,  J.  L.,  Behavior  in  organizations. 
Reading,  Mass.:   Addison-Wesley ,  1973. 

Jaffee,  C.  L.,  Problems  in  supervision:   an  in-basket 
training  exercise.   Reading,  Mass.:   Addison-Wesley,  1968. 

Jaffee,  G.  L . ,  Effective  management  selection:   the  analysis 
of  behavior  by  simulation  techniques.   Reading,  Mass.: 
Addison-Wesley,  1971. 

Kolb,  D.  A.,  Rubin,  I.  M.  and  Mclntyre,  J.  M. ,  Organizational 
psychology- -an  experiential  approach.   Englewood  Cliffs, 
N . J . :   Prentice-Hall,  Inc.,  1971. 

Korman,  A.  K. ,  Industrial  and  organizational  psychology. 
Englewood  Cliffs,  N.  J.:   Prentice -Hal 1 ,  Inc.,  1971. 

Lopez,  F.  M. ,  The  making  of  a  manager.   New  York:   American 
Management  Association,  Inc.,  1970. 

Maier,  N.  R.  F.,  Problem  solving  and  creativity:  in  indi- 
viduals and  groups.   Belmont,  California:   Wadsworth  Publish- 
ing Company,  Inc.,  1970. 

Maier,  N.  R.  F. ,  Solem,  A.  R.  and  Maier,  A.  A.,  Supervisory 
and  executive  development:   a  manual  for  role  playing. 
New  York:   John  Wiley  and  Sons,  Inc.,  1957. 


Meehl,  P.  E.,  Clinical  vs.  statistical  prediction.   Min- 
neapolis:  Univ.  of  Minn.  Press,  1954. 

Murray,  H.  A.,  Explorations  in  personality.   New  York: 
Oxford  Press,  1958. 

Od i o rn e ,  G .  S .  ,  Personnel  administration  by  objectives . 
Homewood,  111.:   Richard  D.  Irwin,  Inc.,  1971. 


OSS  Assessment  Staff,  Assessment  of  men.   New  York:   Rinehart 
and  Co. ,  Inc. ,  1948. 

Salas,  R.  G.,  The  WOSB '  s  revi  steel:   An  experimental  evalu- 
ation of  the  A\  ist  ralian  Army  version  of  the  War  Office 
Sele ct i on  Board  p r ocedtn e  :   Au stralian  Army  Psychological 
Research  Unit,  Melbourne,  1970. 


159 


[101]  Sands,  E.,  How  to  select  executive  personnel.   New  York: 
Reinhold  Publishing  Corp.,  1963. 

[102]  Sechrest,  L.,  Testing,  measuring,  and  assessing  people. 
In  E.  F.  Borgatta  and  W.  W.  Lambert  (Eds.),  Handbook  of 
personality  theory  and  research.   Chicago:   Rand  McNally, 
1968. 

[103]  Stroh,  T.  F.,  Managing  the  new  generation  in  business. 
New  York:   McGraw-Hill,  1971. 

[104]  Tagiuri,  R.  (Ed.),  Research  needs  in  executive  selection. 
Boston:   Harvard  University,  1961 . 

[105]  Thorndike,  R.  L.  Personnel  selection.   New  York:   John 
Wiley  and  Sons,  Inc.,  1949. 

[106]  Vardaman,  G.  T. ,  Effective  communication  of  ideas.   New 
York:   Van  Nostrand  Reinhold  Company,  1970. 

Reports  and  Other 

[107]  Bagley,  D.  H.,  From  the  chief  of  naval  personnel.   The 
Officer  Personnel  Newsletter,  Washington:   Bureau  of 
Naval  Personnel,  October,  1972. 

[108]  Culclasure,  D.  F,,  Development  of  career  motivational 

prediction  and  selection  procedures.   (Report  No.  AFHRL- 
TR -71-19) ,  San  Antonio:   Southwest  Research  Institute, 
January  19  71. 

[109]  Dunne tte,  M.  D.,  Personnel  selection  and  job  placement  of 
the  disadvantaged:   issues,  problems,  and  sug ges tions . 
Technical  Report  No".  4001,  Univ.  of  Minn.,  Aug.  1970. 

[110]  Engel ,  J .  D . ,  An  approach  to  standardizing  human  performance 
assessment .   (HumRRO  professional  paper  26-70),  Oct.  1970. 

[Ill]  England,  G.  W. ,  Agarwal ,  N.  C,  Rydel,  N.  L.,  and  Olsen, 
K .  A . ,  Personal  value  systems  and  their  relationship  to 
administrative  behavior  simulations  and  perceptions  of 
organizational  effectiveness  of  Naval  Officers.   Technical 
report  no.  5004,  Univ.  of  Minn.,  Jan.  1972. 

[112]  Foundation  for  Research  on  Human  Behavior,  Assessing 

managerial  potential.   Report  of  seminar  Feb^Mar  1957. 

[113]  Githens,  W.  H.  and  Elster,  R.  S.,  Subordinate  ratings: 
why  not?   Paper  presented  at  the  15th  Annual  Conference 
of  the  Military  Testing  Association,  San  Antonio,  November 
1973. 


160 


[114]  Ilelme,  W.  H.  ,  Willemin,  L.  P.,  Grafton,  F.  C,  Dimensions 
of  leadership  in  a  simulated  combat  situation .  Te chnic a 1 
Research  Report  1172,  Behavior  and  Systems  Research  Lab., 
Arlington,  VA,  July  1971. 

[115]  Uobbs,  I.  E.,  Rear  Admiral,  USN,  Selection  of  Navy  Headers 
in  the  nuclear  age.   Address  made  at  the  U.  S.  Navy  Manage- 
ment School,  Monterey,  California,  20  Feb  1959. 

[116]  Hunsacker,  P.  L.,  Mudgett,  W.  C.  and  Wynne,  B.  E.,  The 

leadership,  assessment  and  training  simulation:   training , 
assessment  and  research  applications.   Paper  prepared  for 
the  82nd  APA  Convention,  New  Orleans,  Aug-Sep  1974. 

[117]  Keenan,  J.  J.,  et.  al. ,  Concepts  and  practices  in  the 

assessment  of  human  performance  in  Air  Force  systems.  Report 
No.  SSD  65-172 (514)R.   Dunlap  and  Associates,  Inc . ,  Darien, 
Conn.  Sept.  1965. 

[118]  Krumm,  R.  L.,  Robins,  J.  E.  and  Ryan,  T.  G.,  Research  on 

tactica 1  decision  making: 1 1 1 ,  Predictor  variables  and 

criterion  measures.   (Technical  research  note  229)  ,  The 
Bunker- Ram o  Corp.,  May  1973. 

[119]  Lopez,  F.  M. ,  Jr.,  Evaluating  executive  decision  making. 
AMA  Research  Study  75.   New  York:   American  Management 
Association,  1966. 

[120]  McCowan,  W.  B.,  An  evaluation  of  the  usefulness  of  the 
assessment  center  in  personnel  selection  and  promotion . 
XUnpublisKed  thesis,  George  Washington  University) , 
Washington,  D.  C,  197  2. 

[121]  Morsh,  J.  E.,  Survey  of  Air  Force  officer  management 

activities  and  evaluation  of  professional  milita ry  edu c a t i on 
requirements .   (Report  No.  AFHRL-TR-69 -38) ,  Lackland  Air 
Force  Base,  Dec,  1969. 

[122]  Olmstead,  J.  A.,  et .  al . ,  Development  of  leadership  assess- 
ment simulations.   (HumRRO  technical  report  73-21)  Sep.  1975. 

[123]  Ronan,  W.  W.  and  Prien,  E.  P.,  Toward  a  criterion  theory: 

a  review  and  analysis  of  research  and  opinion .   Greensboro, 
N.  C. :   Richardson  Foundation,  1966. 

[124]  Snyder,  Q.  C,  Inventory  and  assessment  at  the  United  States 
Army  War  College .   Paper  presented  at  the  15th  Annual  Con- 
ference of  the  Military  Testing  Association,  San  Antonio, 
November  1973. 

[125]  United  States  Army  Infantry  School  Assessment  Center  Project, 
Staff  Memorandum  of  3  July  1973. 


161 


[126]  United  States  Army  Infantry  School  Assessment  Center  Project, 
Unpublished  report  of  trip  to  British  Royal  Commission  Board 
26  Jan  -  3  Peb  1973. 

[127]  United  States  Army  Infantry  School  Assessment  Center  Project, 
Unpublished  briefing. 

[128]  United  States  Army  War  College,  USAWC  Study  of  Leadership 
for  the  Professional  Soldier,  1  July  1971. 

[129]  United  States  Army  War  College,  Monograph  #1,  Demographic 
Characteristics  of  U.  S.  Army  Leaders.   Leader  ship  ""Mono  - 
graph  Series,  June  1973. 

[130]  United  States  Army  War  College,  Monograph  ii  2 ,    Satisfaction 
With  U.  S.  Army  Leadership.   Leadership  Monograph  Series, 
September  1973. 

[131]  United  States  Army  War  College,  Monograph  #3,  Junior  NCO 
Leadership .   Leadership  Monograph  Series,  October  1973. 

[132]  United  States  Naval  Postgraduate  School,  Superintendent's 
Memorandum  of  15  April  1971  regarding  Planning  Guidance 
for  Fiscal  Years  1973-1978. 

[133]  Weybrew,  B.  B.,  and  Kinsey,  J.  L.,  A  system  analytical 

approach  to  asses  sment  of  human  reliability  .   ( !• !  e  m  o  rep  o  r  t 

no.  68-15),  U.  S~  Naval  Submarine  Medical  Center,  September 
1968. 


3  62 


INITIAL  DISTRIBUTION  LIST 


No.  Copies 


Defense  Documentation  Center  2 

Cameron  Station 
Alexandria,  Virginia   22314 

Library,  Code  0212  2 

Naval  Postgraduate  School 
Monterey,  California  93940 

Assoc.  Professor  R.  S.  Elster,  Code  55Ea  *jq 

Department  of  Operations  Research  and 

Administrative  Sciences 
Naval  Postgraduate  School 
Monterey,  California   93940 

Assoc.  Professor  J.  D.  Senger,  Code  55Se  1 

Department  of  Operations  Research  and 

Administrative  Sciences 
Naval  Postgraduate  School 
Monterey,  California   95940 

CDR  G.  B.  Allen,  USN,  Code  55A1  1 

Department  of  Operations  Research  and 

Administrative  Sciences 
Naval  Postgraduate  School 
Monterey,  California   93940 

Director  1 

Assessment  Center 

U.S.  Army  Infantry  School 

Ft.  Benning,  GA   31905 

Dr.  Donald  Penner  1 

U.S.  Army  War  College 
Carlisle  Barracks,  PA   17013 

Dr.  Cabot  L.  Jaffee  1 

Florida  Technological  University 
Box  25000 
Orlando,  FL   32816 

Library,  Code  55  1 

Naval  Postgraduate  School 
Monterey,  California   93940 


163 


Thesis 

A377H 

c.l 


c 


Ok 


Allen 


23918 

219  06 
25J<,3 


Thesis 

A3774 

c.l 


':llSZ* 


Allen 

The  personnel  assess- 
ment center:   a  review 
and  suggested  applica- 
tion. 


lhesA3774 

The  personnel  assessment  center 

111 


3  2768  001  91016  9 

DUDLEY  KNOX  LIBRARY 


'    A 


<- 


